PaperlayerView exact revision on arXiv

Source-linked AI summary

MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-to-MLM Curriculum

Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk Gozay

arXiv:2608.25768v1cs.CLcs.AI

TL;DR

Turkish encoder pretraining has not systematically tested alternatives to pure MLM. This paper trains MoganBert-TR with a CLM→MLM curriculum and finds decisive gains, including 78.41 on TrGLUE and first-place student performance for its embedding model on MTEB(Turkish).

  • Problem

    Turkish encoder models have modernized their architectures without systematically testing whether pretraining objectives beyond pure MLM work for Turkish.

  • Method

    The paper trains MoganBert-TR with a CLM→MLM curriculum and measures its retrieval advantage through embedding geometry, including reduced variance concentration in a dominant direction.

  • Results

    MoganBert-TR is best among the compared Turkish ModernBERT models on TrGLUE with 78.41, while MoganBert-Embed ranks first among student models on MTEB(Turkish).

  • Takeaways & Limitations

    The study finds that objective choice is decisive for Turkish encoder pretraining and that annealing provides a cheap, effective design space.

  • Takeaways & Limitations

    The objective ablation uses a single seed and short 10,000-step runs, while its retrieval evaluation relies on a single dataset.

Abstract

from arXiv · show

Turkish encoder models have adopted modern architectures while leaving the pretraining objective fixed at masked language modelling. This paper introduces MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on a language-specifically filtered corpus, together with an embedding model derived from it (MoganBert-Embed). MoganBert-TR is trained over 237.3B tokens with a two-stage CLM-to-MLM curriculum: causal language modelling first, masked language modelling for the remainder, with the transition made inside the stable phase of a WSD schedule. In a controlled ablation under an equal step budget, this design outperforms pure MLM by 2.7-3.7x on Turkish MS MARCO retrieval; the measured mechanism is embedding geometry, where a single direction absorbs 28.1% of the variance under pure MLM against 11.9% under the curriculum. Long-context extension and learning-rate decay are then split into two branches after a shared prefix: running the final portion of decay at 1024 context improves the TrGLUE average by 0.49 +/- 0.26 points across five paired seeds (p = 0.013) and beats a model-soup alternative by 0.75 points at ~4.3% additional cost. MoganBert-TR attains 78.41 on TrGLUE, the best among the Turkish ModernBERT models compared, and 77.73 on TabiBench, where it leads two of the eight categories with the largest margin on code retrieval (+3.62 points over TabiBERT). MoganBert-Embed, produced through teacher distillation and multi-signal contrastive fine-tuning, ranks first among student models on the MTEB(Turkish) overall average with 68.30 and reaches 99.5% of its 7.57B-parameter teacher's score with a 51x smaller backbone. The accompanying 50,048-token tokenizer outperforms all compared Turkish tokenizers on compression and fertility across two independent test sets. Weights, tokenizer, embedding model and evaluation code: https://huggingface.co/moganai

1 Introduction

MoganBert-TR examines Turkish encoder pretraining beyond the field’s fixed pure-MLM assumption, using a CLM→MLM curriculum and branched annealing for long-context extension. It reports gains in Turkish retrieval, TrGLUE, embedding quality, and tokenizer performance.

  • Motivation: Turkish encoder work modernized architecture while retaining pure MLM, leaving the objective systematically untested for Turkish.BERTurk, TabiBERT, and ModernBERT-TR brought ModernBERT architecture to Turkish, but shared the fixed-objective assumption.
  • Model and training: 149M-parameter MoganBert-TR is trained from scratch on 237.3 billion tokens with a two-stage CLM→MLM curriculum.The curriculum is evaluated through a controlled ablation using the same architecture and corpus.
  • Training-objective ablation: 2.7–3.7×: CLM→MLM leads pure MLM on Turkish MS MARCO retrieval under an equal step budget.The curriculum and pure-MLM models are practically tied on linear probing, while embedding geometry differs: one direction explains 28.1% versus 11.9% of variance.
  • Branched annealing: +0.49 ± 0.26 points: running final learning-rate decay at 1024 context improves the TrGLUE average across five paired seeds.The branch beats a model-soup alternative by 0.75 points at ∼4.3% additional cost.
  • Evaluation: 78.41 on TrGLUE: MoganBert-TR is best among the Turkish ModernBERT models compared, while scoring 77.73 on TabiBench and leading two categories.On TabiBench, it ranks behind ModernBERT-TR at 77.92 and ahead of TabiBERT at 77.58.
  • Embedding and tokenizer: 68.30: MoganBert-Embed ranks first among student models on MTEB(Turkish) and reaches 99.5% of its 7.57-billion-parameter teacher’s score with a 51× smaller backbone.It is produced through teacher distillation and multi-signal contrastive fine-tuning; the 50,048-tokenizer outperforms compared Turkish tokenizers on two independent test sets.

2 Related Work

Modern encoder improvements have culminated in ModernBERT, which underpins recent Turkish models that otherwise retain pure MLM pretraining. Prior work supports CLM→MLM curricula under fixed compute, while leaving late-phase context length and Turkish quality filtering as open design questions.

  • ModernBERT combines RoPE, alternating local/global attention, GLU activations, unpadding, and FlashAttention to support 8192-token contexts and faster inference.
  • BERTurk established monolingual Turkish BERT, while TabiBERT and ModernBERT-TR brought the ModernBERT architecture to Turkish.BERTurk’s corpus composition and preprocessing remain partly unspecified; TabiBERT also released a standardized evaluation framework.
  • All cited monolingual and multilingual encoder models use pure MLM, which is the pretraining objective MoganBert-TR changes.
  • 87.85 was the sequence-classification average for 25% CLM + 75% MLM, versus 87.00 for 100% MLM, in a 610M-model fixed-compute study.The study evaluated 38 models and more than 15,000 fine-tuning runs, finding CLM→MLM consistently outperformed pure MLM under a fixed compute budget.
  • The role of context length during the final annealing/decay phase had not been measured, motivating the paper’s Section 6.2 experiment.Prior work showed that data presented in the low-learning-rate region is disproportionately influential and that adding low-resource languages there can yield large gains.
  • Turkish corpus construction follows CCNet and FineWeb/FineWeb2 patterns, using WARC extraction, fastText language identification, and sequential quality and repetition filters.FineWeb-Edu demonstrated scalable quality classification from LLM labels, but Turkish lacks an off-the-shelf quality classifier.

3 Pretraining Data

The pretraining corpus combines Turkish web data from FineWeb2 and recent Common Crawl months with domain-dense editorial sources such as books, theses, and academic/legal publications. A single-pass filtering pipeline adds Turkish-specific quality classification, PII masking, and incremental MinHash-based deduplication.

  • Corpus sources: The corpus combines Turkish FineWeb2 data, recent Common Crawl months retrieved through indexed byte ranges, and editorially reviewed books, theses, and academic/legal publications.The web branch avoids downloading complete WARC files, while the second track targets domain-dense content.
  • Filtering and deduplication: The processing chain applies language verification, Gopher/C4-style heuristics, Turkish boilerplate checks, PII masking, and learned quality classification in a single pass.The output is written directly as clean JSONL before deduplication.
  • Filtering and deduplication: MinHash-based fuzzy deduplication uses 5-gram shingles, 14 buckets, and 8 hashes per bucket, incrementally deduplicating each month internally and against the accumulated corpus.This preserves cross-month deduplication while processing the collection incrementally.
  • Turkish quality classification: 70.7% exact and 93.9% ±1 accuracy were achieved by a Turkish BERT quality classifier, but its ~96.5 documents/second throughput was unsuitable for production.The classifier addressed grammatically valid but low-value Turkish text, including gambling, SEO, and advertising.
  • Editorial content: Editorial document processing prioritizes structural classification so covers, copyright notices, prefaces, and tables of contents are excluded from book content.The passage identifies this distinction, rather than transcription accuracy, as the decisive constraint.

4 Tokenizer

The 50,048-token SentencePiece Unigram tokenizer was selected to balance efficiency against embedding cost and outperformed all compared Turkish tokenizers on compression and fertility across two independent test sets.

  • Vocabulary size: 50,048 tokens define the final SentencePiece Unigram vocabulary, chosen as the multiple of 64 closest to 50,000.The tokenizer design and measured efficiency are presented in this section.
  • Vocabulary size: ≈10.7M additional parameters make the 64K vocabulary too costly despite improving fertility from 1.568 to 1.527 over 50K.With hidden size 768, the added parameters represent roughly 7% of a 150M foundation model’s budget, while the fertility difference is 2.7%.
  • Independent evaluation: The Mogan tokenizer outperforms all compared tokenizers on compression and fertility across Turkish Wikipedia and FLORES-200.Both independent test sets used the same document sets for all tokenizers under seed 42.
  • Code handling: 11 multi-space tokens ranging from 3 to 16 spaces preserve indentation in code data under an indentation-preserving pipeline.The pipeline uses the allow_whitespace_only_pieces setting because standard normalization can collapse distinct programs into identical token sequences.
  • Special tokens: Keeping [MASK] in control_symbols prevents literal [MASK] text in web documents from becoming spurious mask identifiers during training.The final special-token layout and rejected alternatives are provided in Appendix B.

5 Model and Pretraining

MoganBert-TR uses a 149.4M-parameter ModernBERT-style Turkish encoder with a CLM-to-MLM curriculum, selected through controlled ablation despite fewer MLM steps. Pretraining additionally adopts token-level random masking and branches late-stage annealing to test longer context at modest extra cost.

  • Architecture: 149.4M parameters comprise a 22-layer ModernBERT-base configuration with 768-wide hidden states, alternating local/global attention, RoPE, and GLU.The implementation rewrote HuggingFace’s bidirectional attention path for CLM, addressing causal windows, packing boundaries, position resets, and document-boundary loss masking.
  • Objective curriculum: 10,000-step ablations selected CLM→MLM: clm25_mlm75 used CLM for ∼2,500 steps and MLM for 7,500, while mlm_only used 10,000 MLM steps.The curriculum therefore used 33% fewer MLM steps, yet differences were obtained in its favour.
  • Objective curriculum: 28.1% of pure-MLM embedding variance concentrates in one direction, versus 11.9% under CLM→MLM, while cosine retrieval separates the models threefold.The paper attributes this contrast to raw embedding geometry: linear probes can ignore dominant directions, whereas cosine retrieval cannot.
  • MLM masking: 41.1% of training tokens belong to words split into ≥2 pieces, so token-level random masking was chosen over separate whole-word or morphology-aware ablations.The token-weighted split rate capped whole-word masking’s distinct coverage at 41%; suffix candidates represented 23.4% of tokens.
  • Annealing and long context: ∼4.3% additional cost enabled branching only the final 4,810 steps, with context changing from 1024 to 8192 while the learning-rate curve remained otherwise identical.Both branches used the same 1-sqrt decay over 9,810 annealing steps; context length was the only branch variable.
  • Objective curriculum: 16.6% of the main run used CLM, compared with the 25% ratio validated in Section 5.3, because full pretraining covered two epochs rather than the ablation regime.The main-run CLM share was 36.0B / 216.7B tokens, and the paper identifies the resulting asymmetry as a limitation.

6 Evaluation

Evaluation uses five-seed TrGLUE for statistically paired comparisons and single-seed, broader TabiBench for coverage. MoganBert-TR leads Turkish ModernBERT models on TrGLUE, while TabiBench shows strong code-retrieval performance but cautions against interpreting sub-point differences as definitive.

  • Evaluation protocols: TrGLUE uses the official script and fixed five-seed protocol, whereas TabiBench covers 28 datasets across eight categories with single-seed measurement.TrGLUE supports paired-seed comparisons; TabiBench measures coverage rather than statistical separation.
  • Annealing and context extension: +0.49 ± 0.26 TrGLUE points favored running final decay at 1024 context across five paired seeds (p ≈0.013), with all seeds positive.The branch was statistically significant and improved held-out MLM loss continuously during annealing, from 1.8026 to 1.6379.
  • Annealing and context extension: +0.75 TrGLUE points and ∼4.3% additional cost made the branching approach preferable to the soup, selecting anneal1k as the final model.The fill-mask soup improved top1 accuracy to 71.8% from parent scores of 69.2% and 64.1%, but that gain did not transfer downstream.
  • TrGLUE results: 78.41 TrGLUE made MoganBert-TR the best compared Turkish ModernBERT model, ahead of TabiBERT at 77.83 and ModernBERT-TR at 77.64.BERTurk remained 1.46 points higher, with the gap concentrated in CoLA and STS-B.
  • Diagnostic analyses: 0.2361 NDCG@10 was the raw model’s score on a ten-set information-retrieval evaluation, motivating the embedding-model work.Anisotropy remained high across MLM encoders at 0.92–0.98, with MoganBert-TR highest; inference retained global_rope_theta=160000.

7 Embedding Model

MoganBert-Embed is produced by correcting MoganBert-TR’s anisotropic geometry through teacher distillation, then combining targeted contrastive signals with Phase 1 in a weighted soup. It ranks first among student models on MTEB(Turkish), reaches 99.5% of its teacher’s score, and trades weaker retrieval for stronger clustering.

  • Phase 1: cos_raw fell from 0.98 to 0.085, while retrieval rose from 0.2361 to 0.5927 at the same step during Phase 1.The change indicates that representations spread beyond the narrow cone and become usable for retrieval.
  • Phase 2: +13.83 points in pair classification and +10.35 in STS followed v3’s targeted losses, while classification improved +2.44 and clustering and bitext regressed.The v3 changes included CoSENT for STS, structured NLI handling, symmetric pair classification, and same-class masking.
  • Weighted soup: 68.30 was achieved by adding Phase 1 to the Phase 2 soup, versus 67.08 for Phase 2 alone; the sweet spot was approximately 0.17.Phase 1 compensated for Phase 2 regressions, raising clustering from 62.19 to 66.46 and bitext from 96.48 to 97.24.
  • MTEB(Turkish) results: 99.5% of the teacher’s score was reached by the 149M-parameter student, trailing its 7.57-billion-parameter teacher by 0.36 points with a 51× smaller backbone.The teacher led retrieval at 64.46 versus 59.58, whereas the student led pair classification, classification, and clustering.
  • MTEB(Turkish) results: 68.30 ranked MoganBert-Embed first among student models, ahead of ModernBERT-TR at 68.13, and it ranked first on 9 of 26 tasks.It led both clustering tasks and the category average by 3.26 points, but ranked first on only three of 11 retrieval tasks and trailed mE5 and BGE-M3 on that category average.

8 Discussion

The discussion identifies pretraining objective, decay-phase context length, and data mixture as independent design levers, while highlighting limits of small probes and current architectures. It attributes code-retrieval gains primarily to mixture share rather than the CLM objective or tokenizer family.

  • Training objective: The objective alone can create a qualitative difference under identical architecture and compute, invisible on classification probes but growing threefold on cosine-based tasks.This suggests encoder comparisons can miss objective-driven differences when relying mainly on classification evaluations.
  • Annealing and context: ∼4.3% of total cost enables branching into two annealing regimes, producing better results than a model soup in these measurements.The discussion identifies context length during decay as comparably decisive to the data mixture.
  • Code retrieval: +3.62 points marks MoganBert-TR’s largest separation from TabiBERT in code retrieval, alongside +6.03 over BERTurk.The discussion considers the indentation-preserving tokenizer and code share as candidate explanations, reporting 10% versus TabiBERT’s 6% code share.
  • Code retrieval: 60.16 for ModernBERT-TR, trained without CLM and with roughly a tenth of its mixture devoted to code, is within half a point of MoganBert-TR’s 60.57.TabiBERT’s 6% code share places it nearly four points below both, supporting mixture share rather than objective or tokenizer family as the co-varying factor.
  • Evaluation limitations: The model soup ranked fifth of six models on TrGLUE despite beating both parents on the fill-mask probe, demonstrating the risk of small-sample checkpoint selection.The discussion characterizes small internal metrics as misleading for model selection.
  • Architectural limitations: BERTurk’s leads on CoLA and STS-B are attributed to an unpretrained [CLS] representation after removing NSP, suggesting an auxiliary objective as a promising unmeasured remedy.All three Turkish ModernBERT models fall behind in the same direction, supporting this architectural-ceiling interpretation.

9 Limitations •

The study’s conclusions are limited by single-seed or short-run evaluations, unvalidated training choices, and benchmark comparability issues. Reproducibility is constrained because the full corpus cannot be released, while some embedding evaluation data lacks human validation.

  • Objective and scale: The objective ablation uses one seed and 10,000-step runs, so its full-pretraining-scale advantage remains unverified; retrieval also uses one dataset and two probes are saturated.The reported ablation advantage therefore requires separate validation at full-pretraining scale.
  • Objective and scale: The 16.6% CLM ratio used in pretraining lacks ablation validation because the ablation tested 25%, and the pure 1024 annealing branch was not measured.These untested settings limit direct attribution of the full-pretraining results.
  • Model soup: Soup components are single-seeded, and MTEB-based weight sweeps introduce overfitting risk, although the flat 0.10–0.25 curve reduces rather than eliminates it.The limitation concerns both seed coverage and selection of soup weights.
  • Benchmark comparability: TabiBench is single-seed, evaluates only MoganBert-TR in this work, and compares runs from different environments, so the 0.19- and 0.15-point gaps are not orderings.The benchmark authors caution that margins below one point may be insignificant; remeasuring all six models in one environment would resolve this.
  • Benchmark comparability: Two reference models were evaluated at 512 tokens versus 2048 for the others, making their long-document retrieval scores lower bounds.The shorter context disadvantages those two reference models on retrieval tasks involving long documents.
  • Embedding evaluation: Embedding evaluation relies on a single teacher, while its sentiment data is LLM-generated and has not been human-validated.Both the teacher choice and the data-generation process limit validation breadth.
  • Reproducibility: Licensing restrictions prevent releasing the full corpus, although the filtering decisions and pipeline code are shared for reproducibility.The unreleased corpus limits complete replication of the training data.

10 Conclusion

The conclusion identifies the CLM→MLM curriculum as decisive for Turkish encoder pretraining and reports strong benchmark performance for MoganBert-TR and MoganBert-Embed. It also outlines future work on retrieval, scaling, and controlled ablations.

  • Main findings: 2.7–3.7×: Under an equal step budget, the CLM→MLM curriculum outperformed pure MLM on cosine-based tasks.The conclusion attributes the mechanism to measured embedding geometry.
  • Main findings: 78.41 on TrGLUE: MoganBert-TR was the best of the Turkish ModernBERT models compared.It reached 77.73 on TabiBench, statistically indistinguishable from ModernBERT-TR (77.92) and TabiBERT (77.58).
  • Main findings: 3.62-point margin: MoganBert-TR exceeded TabiBERT on code retrieval, while MoganBert-Embed ranked first among student models on MTEB(Turkish) with 68.30.The code-retrieval advantage reflects mixture and tokenizer decisions.
  • Future work: Future work targets retrieval-signal weighting, a separate retrieval-focused soup component, ∼400M and decoder variants, and controlled ablations.The proposed ablations include the full-pretraining CLM ratio, pure 1024 annealing, and decomposition of the code-retrieval advantage into tokenizer, mixture, and objective components.

A Data Pipeline Details

The data pipeline indexes Turkish Common Crawl content from monthly Parquet indices, downloads selected WARC records, and applies layered quality, language, boilerplate, and PII filters. Quality-threshold calibration and worker-level resource constraints shaped the production implementation, while an ablation subset totaled approximately 9.4B tokens.

  • Access and indexing: Turkish Common Crawl records are indexed from monthly Parquet files by content_languages, then downloaded from AWS S3 at ∼450–480 records/second using 64 concurrent workers.Each indexed record yields warc_filename, warc_record_offset, and warc_record_length.
  • Filter chain: The filter chain combines WARC parsing, boilerplate removal, Gopher/C4 quality filters, Turkish boilerplate checks, language verification, fastText classification, and PII masking.PII masking covers Turkish national IDs, telephone numbers, e-mail addresses, IBANs, and credit cards via checksum or Luhn verification.
  • Quality classifier: The LLM rubric assigns gambling, casino, and adult content a fixed 0, keeps advertising, real-estate-listing, and SEO content below 3, and empirically retains label-2 documents because most were genuine content.With keep_labels={3,4,5}, discarded label=2 documents were largely news, stockmarket, recipe, and article content.
  • Resource and worker optimization: 16 workers kept memory stable at ∼23–30 GB on a c7i.8xlarge, whereas 24 and 30 workers caused OOM because each worker separately loaded the ∼2.29 GB fastText model.The instance had 32 vCPU and 61 GB RAM.
  • Ablation subset: The ablation subset comprised ∼9.4B tokens from CommonCrawl, Code, FineWeb-edu-eng, Legal, Articles, FinePdfs, Books, FineMath-4Plus, and turkce-matematik.The listed sources contributed 3.63B, 1.70B, 1.25B, 780M, 627M, 588M, 316M, 303M, and 221M tokens, respectively.

B Tokenizer Implementation Details · C Implementation: Four Silent-Failure Pitfalls and Memory

The tokenizer uses parallel normalization and code-specific preprocessing to preserve indentation and line counts, while implementation safeguards address silent data loss, corpus-scale crashes, and memory pressure. The bidirectional architecture was also adapted for causal training with boundary-aware attention, position, and loss handling.

  • B Tokenizer Implementation Details: The shared normalization pipeline orders line-ending normalization before Unicode NFC, soft-hyphen removal, control-character removal, whitespace normalization, and multi-space collapsing.Whitespace and punctuation splitting is delegated to SentencePiece’s pre-tokenization.
  • B Tokenizer Implementation Details: Code preprocessing escapes line endings to preserve indentation and fixed line counts, preventing program collapse and per-line sampling imbalance.The corpus contained 521,000 code files with ∼15–52M lines, versus ∼900K lines from all other sources.
  • B Tokenizer Implementation Details: ∼75% of chunked documents were silently discarded by the 4,192-byte max_sentence_length default, while training crashed at 3.63 billion characters under the default 32-bit node index.The fixes were max_sentence_length=100000 and train_extremely_large_corpus=True; identity normalization does not disable consecutive-space removal.
  • B Tokenizer Implementation Details: 30 tokens represent a five-line Python function when whitespace-only pieces are enabled, versus 46 by default and 26 with lossy indentation removal.For a real 5,612-character code file, tokenization falls from 2,535 to 2,290 tokens.
  • B Tokenizer Implementation Details: The tokenizer assigns [UNK]=0, [PAD]=1, [CLS]=2, [SEP]=3, [MASK]=4, [EOS]=5, unused tokens 6–21, byte tokens 22–277, and learned pieces 278+.Built-in BOS/EOS handling was disabled to avoid duplicate vocabulary roles.
  • C Implementation: Four Silent-Failure Pitfalls and Memory: Causal mode requires a (w −1, 0) local window, while bidirectional mode reverts to (w/2, w/2), because a symmetric causal window leaks future tokens.The implementation also prevents attention from crossing packed document boundaries and resets RoPE positions for each document.
  • C Implementation: Four Silent-Failure Pitfalls and Memory: −100 labels are required at each packed document’s final position because attention boundary cuts do not prevent shifted CLM loss from predicting the next document’s first token.This masking is separate from cu_seqlens-based attention segmentation.
  • C Implementation: Four Silent-Failure Pitfalls and Memory: ∼9.8 GB of bf16 logits arises from 98K × 50,048 positions, while checkpointing reduces peak logits memory to ∼1.6 GB by recomputing chunks during backpropagation.Plain chunking still retains every chunk’s logits for autograd; .float() and log_softmax each add a further 19.5 GB.

D TabiBench Task-Level Results · E MTEB(Turkish) Task-Level Results

The paper reports task-level results for all 28 TabiBench tasks and eight category averages, alongside all 26 MTEB(Turkish) tasks. TabiBench evaluation includes reproducibility corrections and consistent scoring procedures, while MTEB compares student models with a teacher model.

  • D TabiBench Task-Level Results: 28 TabiBench tasks and eight category averages are reported, with category averages weighted by test-set size.The table excludes mmBERT from ranking and marks ModernBERT-TR per-task scores as “?” because it publishes results only at category level.
  • D TabiBench Task-Level Results: TabiBench rankings bold the best row score among monolingual Turkish models, while mmBERT is excluded following the source work.Scores for TabiBERT, BERTurk, YTU-Cosmos-BERT, TurkishBERTweet and mmBERT come from the cited reference tables.
  • D TabiBench Task-Level Results: Three reference-loader defects were fixed before measurement to prevent silent score distortion.Corrections covered WikiNER label mapping and were documented for reproducibility.
  • D TabiBench Task-Level Results: 28 runs completed with zero exit status, with category membership matching the reference work and no task counted twice.The evaluation also used consistent score scales: QA on 0–100 and other tasks on 0–1, multiplied by 100 in tables.
  • E MTEB(Turkish) Task-Level Results: 26 MTEB(Turkish) tasks are reported with the best student-model score bolded in each row.The rightmost column contains the teacher model.
  • E MTEB(Turkish) Task-Level Results: MTEB(Turkish) task-level results distinguish student-model rankings from the teacher-model reference column.The table’s rightmost column is explicitly designated for the teacher model.
Loading 2608.25768v1…