PaperlayerView exact revision on arXiv

Source-linked AI summary

T-pro 2.0: An Efficient Russian Hybrid-Reasoning Model and Playground

Dmitrii Stoianov, Danil Taranets, Olga Tsymboi, Ramil Latypov, Almaz Dautov, Vladislav Kruglikov, Nikita Surkov, German Abramov, Pavel Gein, Dmitry Abulkhanov, Mikhail Gashkov, Viktor Zelenkovskiy, Artem Batalov, Aleksandr Medvedev, Anatolii Potapov

arXiv:2512.10430v1cs.CL

TL;DR

Russian-language reasoning research lacks a unified open ecosystem, including scarce evaluation sets and public demos for comparing reasoning modes. T-pro 2.0 combines an open-weight hybrid-reasoning model and adapted inference pipeline; it reports strong results on Russian reasoning benchmarks, while dedicated agentic improvements remain outside its scope.

  • Problem

    Russian LLM research has few open strong models, high-quality evaluation sets, and public demos for comparing direct answers, reasoning traces, and inference optimizations.

    Source: §1 · p. 1

  • Method

    T-pro 2.0 combines an open-weight Russian model with direct-answer and reasoning modes, Cyrillic-dense tokenizer adaptation, instructional midtraining, and EAGLE-style speculative decoding.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    On ruAIME 2024 and 2025, T-pro 2.0 scores 0.704 and 0.646, outperforming DeepSeek-V3, GPT-4o, and all proprietary Russian models.

    Source: §4.2 · p. 6

  • Takeaways & Limitations

    Its direct-answer and reasoning modes let applications balance speed and accuracy within a single deployed system.

    Source: §1 · p. 1

    The model has no dedicated improvements for tool use or agentic behavior, and performance in those areas is expected to be comparable to or slightly below Qwen3-32B.

    Source: Limitations · p. 7

Abstract

from arXiv · show

We introduce T-pro 2.0, an open-weight Russian LLM for hybrid reasoning and efficient inference. The model supports direct answering and reasoning-trace generation, using a Cyrillic-dense tokenizer and an adapted EAGLE speculative-decoding pipeline to reduce latency. To enable reproducible and extensible research, we release the model weights, the T-Wix 500k instruction corpus, the T-Math reasoning benchmark, and the EAGLE weights on Hugging Face. These resources allow users to study Russian-language reasoning and to extend or adapt both the model and the inference pipeline. A public web demo exposes reasoning and non-reasoning modes and illustrates the speedups achieved by our inference stack across domains. T-pro 2.0 thus serves as an accessible open system for building and evaluating efficient, practical Russian LLM applications.

1 Introduction

Russian LLM research lacks a unified open ecosystem for studying reasoning, with scarce high-quality evaluations and few public demos for comparing reasoning modes or inference optimizations. T-pro 2.0 addresses this with an open-weight hybrid-reasoning model, supporting datasets and benchmark, and an interactive demo.

Motivation

  • Russian open models are typically small multilingual adaptations, while strong models are often closed, and high-quality evaluation sets and public reasoning demos remain scarce.The gap includes few demos for comparing direct answering with step-by-step reasoning or observing inference-time optimizations.Source: §1 · p. 1

Model

  • T-pro 2.0 is an open-weight Russian LLM offering direct-answer and explicit-reasoning modes within one system.Its training combines a Cyrillic-dense tokenizer, instructional midtraining, supervised fine-tuning, preference optimization, and adapted EAGLE-style speculative decoding.Source: §1 · p. 1

Research resources

  • T-Wix is an approximately 500k-sample Russian hybrid-reasoning SFT dataset, and T-Math evaluates curriculum-aligned reasoning on Russian high-school olympiad mathematics.Source: §1 · p. 1

Demonstration

  • The interactive demo supports side-by-side reasoning and non-reasoning comparisons, runs tasks from the datasets and benchmarks, and displays inference-time optimization telemetry.The model components and benchmark use Apache-2.0; T-Wix uses ODC-By.Source: §1 · pp. 1–2

2 Related Work

Prior Russian LLM work spans Russian-centric pretraining and multilingual-model adaptation, while instruction datasets and publicly documented speculative-decoding deployments remain limited. T-pro 2.0 sits within this landscape of open Russian models and reasoning resources.

Russian LLMs

  • Russian LLM development spans Russian-centric pretraining and adaptation of multilingual models, with early Russian models facing a capability gap against leading multilingual LLMs.T-pro 1.04 continued pretraining on Russian corpora and achieved state-of-the-art MERA results among open Russian models.Source: §2 · p. 2

Russian Instruction Datasets

  • Russian instruction datasets vary in provenance and domain coverage, but are usually small and contain few reasoning-intensive tasks.Examples include self-instruct Saiga, coding and general-knowledge sources in GrandMaster-PRO-MAX, and translated and native samples in RuAdapt.Source: §2 · p. 2

Efficient Inference

  • Speculative decoding and multi-token prediction offer inference-efficiency approaches, but speculative decoding remains underexplored for general-purpose Russian LLMs.EAGLE uses a lightweight head to draft token trees for parallel verification and is reported to achieve 2–3× speedup.Source: §2 · p. 2

3 T-pro 2.0

T-pro 2.0 combines an interactive demo with a training pipeline that adapts the tokenizer, midtrains and post-trains the model, and adds speculative decoding. The demo exposes model comparisons and inference telemetry, while the training recipe targets Russian-language segmentation, instruction following, and reasoning.

3.1 System and Demonstration Description

  • The demo serves T-pro 2.0 with an EAGLE-style draft-head and 32B verifier on one H100, alongside Qwen3-32B with standard decoding on another.The deployment supports around 20 concurrent users per model.Source: §3.1 · p. 2
  • Users can compare parallel completions, choose standard or reasoning modes, and inspect streamed outputs, decoding settings, domain prompts, and per-request performance telemetry.Telemetry includes generated tokens, end-to-end latency, streaming throughput, and T-pro 2.0 speculative-token acceptance ratio.Source: §3.1 · pp. 2–3

3.2 Training recipe

  • The tokenizer replaces 34k low-frequency non-Cyrillic Qwen3 tokens with Cyrillic tokens while preserving vocabulary size.Its expansion set draws on four tokenizers, and refinement passes make approximately 95% of candidates reachable.Source: §3.2 · pp. 3–4
  • On Russian Wikipedia, the share of words tokenized in at most two tokens rises from 38% to 60%, with shorter average segmentations across all eight evaluated Cyrillic languages.Source: §3.2 · p. 4
  • Instructional midtraining adapts Qwen3-32B to the tokenizer and reasoning on 40B tokens, with Russian and English making up 49% and 36% of the mixture.In an ablation, instruct-only midtraining raises ruAIME 2024 from 0.60 to 0.67; tokenizer experiments report MERA macro-average 0.574 versus 0.560 for the original Qwen3 tokenizer.Source: §3.2 · p. 4
  • T-Wix post-training uses 468k filtered general instructions and approximately 30K translated reasoning samples, then applies DPO to 100k instructions with a 90/10 general-to-reasoning ratio.The reward model uses a Bradley–Terry preference objective and tournament-generated comparisons; DPO pairs are formed from 16 RM-scored completions per instruction.Source: §3.2 · pp. 4–5
  • At temperature 0.8, EAGLE speculative decoding achieves 1.85× average speedup in standard mode, with STEM at 1.99× and humanities at 1.62×.A lightweight draft model proposes candidate tokens for verification by the frozen target model, using EAGLE-2 dynamic draft trees via SGLang.Source: §3.2 · p. 5

4 Evaluation

The evaluation covers Russian knowledge, dialogue, and reasoning, including a new olympiad-level math benchmark. T-pro 2.0 scores strongly across Russian tasks and remains competitive on English reasoning benchmarks.

4.1 Benchmarks

  • T-Math comprises 331 automatically extracted and human-verified problems from All-Russian and Moscow olympiads spanning 1998–2025.The evaluation also includes factual knowledge, dialogue, and reasoning benchmarks, including MERA, Arena Hard Ru, AIME, and localized English reasoning suites.Source: §4.1 · p. 5

4.2 Results

  • T-pro 2.0 scores 0.66 on MERA and 0.697 on ruMMLU-Pro, close to GPT-4o on ruMMLU-Pro (0.714) and above RuadaptQwen3-32B-Instruct (0.652).Table 4 compares Russian-language understanding and dialogue benchmarks; Arena Hard 2 includes Hard Prompt and Creative Writing subsets, with paired values denoting reasoning and non-reasoning settings.Source: §4.2 · p. 5, Table 4 · p. 6
  • T-pro 2.0 reaches 91.1 on Arena Hard Ru and 72.6 on WildChat Hard Ru, outperforming all open-source systems and most proprietary ones.On Arena Hard 2, it scores 53.5 on Hard Prompts and 64.8 on Creative Writing.Source: §4.2 · p. 5, Table 4 · p. 6
  • T-pro 2.0 scores 0.541 on T-Math, 0.704 and 0.646 on ruAIME 2024 and 2025, and 0.94 on ruMATH-500.The results also characterize T-Math as challenging, revealing differences obscured by translated or adaptation-based alternatives.Source: §4.2 · p. 6, Table 5 · p. 6
  • On English reasoning benchmarks, T-pro 2.0 scores 0.765 on AIME 2024, 0.966 on MATH-500, and 0.556 on LiveCodeBench, on par with or better than similar-scale open models.The paper reports that Russian-focused training does not hurt English performance.Source: §4.2 · p. 6

5 Conclusion

T-pro 2.0 combines Russian-focused reasoning adaptation with efficient inference in an open-weight model. Its authors present targeted adaptation as a practical route for resource-limited languages and release supporting datasets and a benchmark.

5 Conclusion

  • T-pro 2.0 combines a Cyrillic-dense tokenizer, reasoning-focused midtraining, and adapted EAGLE-style speculative decoding to deliver strong Russian-task performance without increasing model size or notable English-benchmark degradation.The release includes T-Wix, an SFT dataset enriched with reasoning traces, and T-Math, a Russian mathematical and analytical benchmark.Source: §5 · p. 7
  • The authors frame targeted adaptation of strong multilingual backbones as a practical, reproducible route to high-quality models for languages with limited resources.They also identify tokenizer design and inference optimization as key components for deploying reasoning-capable models beyond English.Source: §5 · p. 7
  • The released models, datasets, evaluation code, and public demo are intended to support Russian-language reasoning research and more transparent, consistent evaluation.Source: §5 · p. 7

Ethical Statement

The ethical statement identifies potential misuse and data-quality risks, describes mitigation measures, and notes that the work does not involve human subjects or personally identifiable information.

Possible Misuse

  • Without appropriate safeguards, the work may enable misleading, offensive, or otherwise harmful content.The authors do not support applications that restrict access to information, facilitate disinformation, target people or groups, or automate harmful actions.Source: Ethical Statement · p. 7
  • Post-training toxicity and safety filtering and usage guidelines are presented as risk-mitigation measures.Source: Ethical Statement · p. 7

Biases and Data Quality

  • Public Russian and English training data may contain stereotypes, factual inaccuracies, or cultural biases, and residual biases may remain despite filtering and manual checks.The authors recommend additional evaluation when transferring the model to domains or communities underrepresented in the training data.Source: Ethical Statement · p. 7

Human Subjects and Privacy

  • The work involved neither human-subjects research nor collection of personally identifiable information, and its data sources comply with their licenses and usage policies.Source: Ethical Statement · p. 7

Limitations

The section identifies limits in agentic capability, reinforcement learning, long-context validation, and reproducibility, and lists the released resources’ licenses.

Limitations

  • No dedicated tool-use or agentic-behavior improvements were made, so performance in these areas is expected to be comparable to or slightly below Qwen3-32B.Source: Limitations · p. 7
  • Alignment used offline DPO only; without interactive feedback, robustness and out-of-domain performance may be limited.Source: Limitations · p. 7
  • Training used a fixed 32k-token context window; coherence and retrieval at the theoretically supported 128k tokens have not been empirically validated.Source: Limitations · p. 7
  • Full reproducibility is restricted because proprietary datasets were used in midtraining and the DPO stage.Source: Limitations · p. 7

A Released Resources and Licenses

B Tokenizer adaptation statistics

This section describes tokenizer-compression gains across Russian and other Cyrillic languages and the data and training processes used for Russian-language model adaptation. It covers T-Wix construction and reasoning-data selection, supervised training and preference tuning, and reward-model data generation and evaluation.

C Instructional midtraining

  • Midtraining continues Qwen3-32B with 40B instruction-style tokens to adapt it to the denser Russian tokenizer and strengthen Russian language and reasoning without degrading base capabilities.Loss falls sharply over roughly the first 1k steps before plateauing; the authors report no additional ablations on answer regeneration or InsTag deduplication.Source: C Instructional midtraining · pp. 12, 15, Figure 3 · p. 14

B Tokenizer adaptation statistics

  • T-pro 2.0 reduces Russian tokens per word and increases the share of words tokenized into at most two tokens, while English compression is essentially unchanged; it lowers tokens per word across eight Cyrillic languages and leads seven of eight tokenizer comparisons.The gains are particularly large for Kyrgyz, while T-pro 2.0 remains competitive on Kazakh.Source: B Tokenizer adaptation statistics · p. 12, Table 7 · p. 13, Table 8 · p. 13

C Instructional midtraining

  • For an 8B model, T-pro 2.0 scores 0.574 versus 0.560 for original Qwen3 on MERA, with small per-task differences; instruct-only midtraining matches or beats the mixed variant on most math and reasoning benchmarks at the same token budget.Source: Table 13 · p. 15, C Instructional midtraining · p. 15

D.1 General part of T-Wix

  • The general T-Wix portion contains 468k diverse, deduplicated prompts from open-source data and high-quality English-dataset translations, targeting coding, mathematics, dialogue, and other LLM capabilities.Source: §D.1 · p. 15

D.1.1 Data Preparation

  • The 14M-instruction SFT preparation pipeline filters and balances samples, then generates completions with top LLMs and rejection sampling to produce a diverse, high-quality dataset without benchmark leakage or redundancy.The final responses are selected from generated candidates using reward-model scores.Source: §D.1.1 · pp. 15–16

D.2 Long Context

  • The long-context SFT increment draws public texts across domains and 8k–32k-token lengths, with generated summarization, QA, and reasoning tasks.It contributes about 1% of SFT samples and 7.7% of tokens.Source: §D.2 · p. 17

D.3 Parallel Corpora

  • Experiments on parallel English data identify approximately 10% of the total SFT mix as the optimal English share for maintaining English proficiency.Source: §D.3 · p. 17

D.4 Reasoning part of T-Wix

  • The reasoning subset starts from about 450k English instructions translated into Russian and deduplicated, then applies reward-based filtering of teacher and student generations to select tasks suited to student learning.Instructions with unstable reward scores or consistently poor student scores are excluded; mean teacher and student rewards estimate the reasoning gap.Source: §D.4 · pp. 17–18
  • Approximately 30k reasoning samples are selected to balance task difficulty with learning potential, with 90% Russian and 10% English instructions.Samples are retained when student–teacher mean-reward gaps are moderate; verifiable tasks use the highest-reward correct completion, while open-ended tasks use the shortest trace among the top three reward-ranked completions.Source: §D.4 · p. 18

D.5 T-Wix dataset analytics

  • 500k samples make up T-Wix, whose token distribution is presented in Figure 4.The figure’s token counts were computed with tiktoken using the o200k base tokenizer.Source: §D.5 · p. 18, Figure 4 · p. 18

D.6 SFT Training Recipe

  • SFT training took 9 hours on four nodes with 8×H100 GPUs, using gradient checkpointing, FSDP, and packed 32k-token contexts without truncation.The selected hyperparameters are listed in Table 14.Source: §D.6 · p. 18, Table 14 · p. 18

E Preference tuning

  • On-policy DPO forms preference pairs from the highest- and lowest-scoring of 16 SFT-model completions per instruction and improves alignment, coherence, and reasoning structure while preserving general capabilities.The 100k-pair dataset uses filtered T-Wix data and cross-subset augmentation: general examples are reformulated in reasoning style and reasoning examples in general style.Source: E Preference tuning · p. 19

E.1 Preference Training Recipe

  • Preference training took 28 hours on four nodes with 8×H100 GPUs, using sequence parallelism.The selected hyperparameters are listed in Table 15.Source: §E.1 · p. 19, Table 15 · p. 19

F Reward Model

  • Tournament-based preference generation reduces comparison requirements while retaining informative preference signals, and trains a Qwen3-32B reward model with a regression head.An external LLM judges matchups between similarly grouped models; pairs are evaluated in both orders and position-biased samples are excluded. Training uses the Bradley–Terry formulation.Source: F Reward Model · pp. 19–20
  • The reward model achieves the highest ∆BoN on Arena-Hard-RU, while performing comparably to open-source models on translated RewardBench 2.The Best-of-N benchmark uses N = 8; ∆BoN measures the difference between win rates for best@8 and worst@8 selections.Source: F Reward Model · p. 20, Table 16 · p. 20
  • The Google Vertex prompt yields superior evaluation quality across benchmarks, particularly on RewardBench 2 (RU).Prompting strategies were drawn from JudgeBench.Source: F Reward Model · p. 20

F.1 Reward Model Analysis

The reward-model analysis compares judge choices and studies transitive preference samples and completion-length effects on RewardBench 2 (RU).

F.1 Reward Model Analysis

  • DeepSeek-V3 shows stronger judgment on open-domain and chat tasks, while Qwen3-235B-A22B performs better in math, code, and other domains.Source: §F.1 · p. 20, Table 17 · p. 21
  • Removing transitive preference samples consistently degrades evaluation metrics, while adding samples beyond the first closure yields marginal positive gains.Source: §F.1 · p. 20, Table 17 · p. 21
  • Skewing chosen-versus-rejected completion lengths lowers RewardBench 2 (RU) quality; Qwen3-235B-A22B as judge favored longer completions regardless of semantic quality.Source: §F.1 · p. 21

G Speculative Decoding Implementation

The implementation adapts EAGLE speculative decoding for T-pro 2.0, training a lightweight draft model and deploying dynamic draft trees to reduce inference latency.

G Speculative Decoding Implementation

H T-Math benchmark

T-Math8 is a 331-task Russian high-school olympiad benchmark designed for automatic evaluation of long-chain mathematical reasoning. It remains challenging for strong reasoning models, none of which exceeds 0.75 pass@1.

H T-Math benchmark

  • T-Math8 contains 331 single-answer problems from Russian school and Moscow Olympiads in mathematics from 1998–2025.Numeric gold solutions make the benchmark suitable for automatic evaluation of long-chain mathematical reasoning.Source: H T-Math benchmark · p. 22
  • Questions and answers are extracted from PDFs with Qwen2.5-VL-72B-Instruct and manually reviewed against the original olympiad sources.The review covers both question texts and verifiable answers.Source: H T-Math benchmark · p. 22
  • Filtering removes multi-answer, non-unique, theorem-style, non-numeric, and auxiliary-figure tasks, plus medium-difficulty items Qwen3-8B nearly solves at pass@16.The filtering targets challenging instances with verifiable answers.Source: H T-Math benchmark · p. 22
  • Evaluation standardizes final answers in boxed format and uses math_verify to compare predicted and reference expressions.Source: H T-Math benchmark · p. 22
  • Table 24 reports pass@1 scores for several strong reasoning models; frontier systems including o4-mini-high, DeepSeek-R1, and Gemini 2.5 Pro are competitive, yet none exceeds 0.75 pass@1, so the benchmark remains far from saturated.Source: H T-Math benchmark · p. 22

I Additional Evaluations

T-pro 2.0 retains strong English reasoning performance despite its primary Russian optimization. On advanced English benchmarks, it remains close to Qwen3-32B and outperforms a reasoning-distilled system on several metrics.

I Additional Evaluations

  • On English advanced reasoning benchmarks, T-pro 2.0 stays close to Qwen3-32B, slightly improves MATH-500, and beats DeepSeek-R1-Distill-Qwen-32B on several metrics.Margins remain narrow on AIME 2024/2025 and GPQA.Source: I Additional Evaluations · pp. 22–23, Table 23 · p. 23
  • The authors report that the Cyrillic-focused tokenizer and training pipeline do not meaningfully degrade English performance, maintaining robust cross-lingual generalization with minimal loss on advanced benchmarks.Source: I Additional Evaluations · p. 23
Loading 2512.10430v1…