K2-V2: A 360-Open, Reasoning-Enhanced LLM

Overview

• Figure 1 contrasts K2’s base-model performance across reasoning scenarios with K2 Instruct’s comparison to larger reasoning-tuned models.The caption identifies Olmo 3.1 32B as a recent fully open-source reference model.Source: Figure 1 · p. 1
• The paper lists Hector Liu and Eric Xing as correspondence contacts.Their email handles are provided in the correspondence line.Source: Overview · p. 1

1 Introduction

• K2 combines broad knowledge, long-context consistency, mathematical knowledge, and reasoning behaviors in a dense 70-billion-parameter model.The authors describe it as competitive with the best open-weight models of its size class and as a robust base for further post-training.Source: §1 · p. 3
• The authors argue that powerful models increasingly hide data composition, infrastructure, and training dynamics, while weak bases offer limited research and production utility.They cite reasoning tuning on weak base models as yielding little scientific insight.Source: §1 · p. 3
• K2’s documented mid-training stages and data compositions support targeted continuous training, including data replay to help maintain base capability.The authors connect this transparency to adaptation without the trial-and-error associated with closed models.Source: §1 · p. 3
• K2 offers researchers a transparent testbed for examining how complex mathematical reasoning and agentic workflows emerge.The introduction says access to training corpora also helps connect post-training outcomes to pre-training signals.Source: §1 · pp. 3–4
• Figure 2 frames K2’s training phases as progressively enabling specific capabilities while addressing phase-specific challenges.The figure caption does not specify the capabilities or challenges for individual phases.Source: Figure 2 · p. 4
• The release adds TxT360-Midas and TxT360-3efforts alongside full development-cycle artifacts to reduce repeated reproduction work.The authors describe Midas as supporting reasoning behaviors and longer contexts, and 3efforts as a mixed-reasoning-effort SFT dataset.Source: §1 · pp. 3–4

2 Overview

• The capability-first strategy targets broad and deep knowledge, long-context consistency, early exposure to reasoning behaviors, and native tool calling.The overview presents these as five pillars, with broad knowledge serving as a linguistic foundation and domain knowledge emphasizing mathematics, programming, and scientific literature.Source: §2 · pp. 4–5
• The capability targets include maintaining coherence over extended sequences and using reasoning behaviors such as planning, deduction, and backtracking as core primitives.Tool use is framed both as support for assistant tasks and as a way to delegate precise work such as calculation or code execution.Source: §2 · pp. 4–5
• Pre-training builds a broad knowledge base through diverse natural data, with scaling focused on training dynamics, infrastructure, and token volume.The paper describes the phase as combining training-dynamic design and monitoring, large-scale infrastructure, and data scaling.Source: §2 · p. 5
• Mid-training specializes the model, extends context, and exposes it to reasoning while addressing compute bottlenecks, distribution drift, and scarce long-context data.The authors use context parallelism, precise token mixtures, synthetic data, and a low constant learning rate to address those challenges and stabilize training.Source: §2 · p. 5
• A final, simple SFT stage injects tool use and finalizes behavior with a focused, high-value corpus to demonstrate the base model’s post-training potential.The overview says this stage confirms the effectiveness of the preceding phases, but does not claim to unlock the model’s upper bound.Source: §2 · p. 5
• The paper aligns its organization with the training lifecycle, examining data composition, infrastructure, and machine-learning dynamics across phases.Its roadmap covers pre-training, mid-training, base-checkpoint evaluation, SFT, final benchmarks, and qualitative capability development.Source: §2 · pp. 5–6
• The overview identifies qualitative study of capability development over time as part of the paper’s evaluation coverage.The paper’s roadmap places this study in Section 8.Source: §2 · p. 6

3.1 Model Architecture

• K2 is an 80-layer decoder-only dense transformer with hidden dimensionality 8,192 and feed-forward intermediate size 28,672.These dimensions specify the model’s depth and principal widths.Source: §3.1 · p. 6
• Its attention has 64 heads of dimension 128, with 8 shared key-value heads using grouped-query attention.The authors adopt grouped-query attention to improve memory and compute efficiency during attention computation.Source: §3.1 · p. 6
• The model applies RoPE with θ = 500,000 and RMSNorm with epsilon 10^-5.Source: §3.1 · p. 6

• The authors jointly optimize batch size, learning rate, and weight decay, characterizing AdamW dynamics with an effective averaging timescale τepoch and scaling its target across model sizes.Learning rates are swept at small scale and extrapolated using 1/dmodel scaling.Source: §3.2 · p. 6
• The recipe targets the batch size that minimizes final validation loss at a fixed token budget, then adjusts it to accelerator constraints rather than targeting the critical batch size.The authors prioritize sample efficiency over step efficiency for data-limited runs.Source: §3.2 · p. 6
• 2.815 for cosine decay-to-zero, 2.845 for cosine decay-to-10%, and 2.817 for linear decay-to-zero on SlimPajama validation showed negligible pilot differences between schedules.The ablation used a 257M-parameter pilot model trained on 41.48 × 10^9 tokens.Source: §3.2 · p. 7
• The final run uses η = 1.5 × 10−4, a global token batch of 9.8 × 10^6, 1.25 × 10^6 steps, and λ = 0.05.The 1200-sequence batch was adjusted from the scaling-law suggestion of 1136 sequences to satisfy hardware constraints.Source: §3.2 · p. 7
• A 1.5 × 10−6 learning-rate floor replaced cosine decay-to-zero after loss and parameter norms became stale below 1% of peak; the floor prevented long-tail stagnation.The floor was held constant for the remainder of pre-training and simplified the transition into mid-training.Source: §3.2 · p. 7

3.3 Pre-training Data: TxT360

• Around 12 trillion tokens of pretraining combine natural-text grounding with global deduplication that enables quality-based upsampling.Document persistence is used as a simple quality signal for deciding what to upsample.Source: §3.3 · p. 7

3.3.3 Papers

• About 107 billion words of papers from ArXiv, PubMed, S2ORC, and PhilPapers receive source-specific processing and filtering.ArXiv LaTeX is converted to Markdown while preserving tables, equations, and section headers; figures and references are removed.Source: §3.3 · p. 8, §3.3.3 · p. 11

3.3.1 English Web

• 3.5 trillion words across 6.4 billion documents make TxT360-CC the largest corpus component; its BestOfWeb subset retains about 22% of documents.BestOfWeb applies the ProX document filter together with a format score for features such as headings, tables, and code blocks, allowing this high-quality subset to be upsampled flexibly.Source: §3.3.1 · pp. 8, 10

3.3.2 Synthetic QA

• Over 95% average support scores across 8,000 sampled documents validate the synthetic QA pairs, which vary in difficulty and are appended to their source documents.Manual checks found that low-quality questions were generic or weakly grounded, while high-quality questions targeted concrete attributes answerable from a short passage span.Source: §3.3.2 · p. 11

3.3.4 Math

• 344 billion tokens of math-focused corpora include MegaMath-Web, which preserves equations and symbols, and MegaMath-Code, filtered for math- and science-relevant code.MegaMath-Web contributes approximately 279B tokens and MegaMath-Code approximately 28.1B; the corpora are decontaminated against 12 downstream benchmarks, removing 0.01% of documents.Source: §3.3.4 · p. 12

3.3.5 Code

• RefineCode is supplied as original files, a Fill-in-the-Middle variant, and a repository-level topologically sorted variant from the same underlying GitHub repositories.The sorted form orders files by import dependencies so definitions precede uses; the FIM form changes file layout while retaining its code content.Source: §3.3.5 · pp. 13–14

3.3.6 Multilingual

• Over 203 billion words of Arabic text across approximately 498 million documents are complemented by multilingual resources including aligned Europarl proceedings.Europarl aligns English sentences with translations across multiple European languages while preserving semantic equivalence.Source: §3.3.6 · p. 14
• The multilingual data configuration lists Finnish, German, Dutch, Greek, Italian, French, English, Portuguese, Swedish, Danish, and Spanish.Source: §3.3.6 · p. 14

3.3.7 Other High-Quality Data Sources

• TxT360 adds structured high-quality sources including FreeLaw, StackExchange, USPTO, PG-19, HackerNews, and Ubuntu IRC, applying language, length/character, and unigram-probability filters.These sources use formats such as XML, LaTeX, PDF, and CSV, and require source-specific processing.Source: §3.3.7 · pp. 14–15

3.3.8 Pre-training Data Analysis

• BERTopic groups Common Crawl documents into 17 broad topics, merging 128 seed groups manually to compare structural and lexical attributes across topics.The topic distribution is summarized in Figure 6.Source: §3.3.8 · pp. 15–16, Figure 6 · p. 16
• Deduplication enables controllable upsampling and reduces train–test overlap, contamination, memorization, and risks from repeated-data double descent.The authors use deduplication and selective upsampling to control the pre-training distribution rather than rely on source-dataset distributions.Source: §3.3.8 · p. 16
• Near-deduplication spans 99 Common Crawl snapshots and curated data, using cleaned-text 13-grams and 128-permutation MinHash; representatives favor curated and newer documents.The process uses Dask for scalable processing.Source: §3.3.8 · pp. 16–17
• Smaller near-duplicate clusters mostly capture near-exact variants or incrementally edited documents, while clusters of 1,000 or more tend to reflect mass replication of generic templates.Examples of large-cluster text include disclaimers and terms and conditions.Source: §3.3.8 · p. 17

3.3.9 Data mix

• Common Crawl documents receive heuristic duplicate-count weights of 3, 5, 8, or 10 across duplicate ranges 2–5, 5–100, 101–1000, and over 1000; repeated non-Common-Crawl documents receive 2.The thresholds are heuristic and informed by preliminary small-scale experiments.Source: §3.3.9 · p. 17, Table 2 · p. 18

3.4 Pre-training Infrastructure

• Hybrid parallelism shards computation across GPUs so each can hold model parameters, optimizer states, gradients, and activations in HBM.The framework is based on Megatron Core and uses Transformer Engine as its backend.Source: §3.4 · p. 19

3.4.1 Hardware and Cluster Setup

• The H200 cluster pairs eight 141 GB GPUs per server with 400 Gb/s InfiniBand, while submitter detects faults and automates job recovery.Each server’s eight GPUs operate at 700 W TDP.Source: §3.4.1 · p. 19

3.4.2 Parallelism and Training Infrastructure

• Data parallelism requires a complete model replica per worker, tensor parallelism adds cross-device communication, and pipeline parallelism can leave GPUs idle.FSDP can reduce memory use with marginal communication overhead, but Megatron Core lacked full FSDP support when the system was set up.Source: §3.4.2 · p. 19
• K2 pre-training uses tensor, sequence, and data parallelism, with pipeline parallelism disabled because the small per-GPU batch would cause pipeline bubbles and hurt utilization.Sequence parallelism is important for the 8K context setup, and activation recomputation mitigates GPU memory spikes.Source: §3.4.2 · pp. 19–20

3.5.1 The Effect of Decay2Zero

• Below a learning rate of 1.5 × 10−6, the parameter norm ceases changing; the authors hypothesize training stagnation and conclude pre-training at a non-zero rate.From steps 1.23e6 to 1.25e6, D2Z’s parameter-norm change slows while the 1%-tail schedule continues to decrease.Source: §3.5.1 · p. 20, Figure 8 · p. 21

3.5.2 Training monitoring

• The loss-spike detector requires both a sustained-floor condition and a severity-peak condition within a sliding window, filtering transient fluctuations and stable high plateaus.The system uses two sensitivity tiers: an alert tier and a stricter restart tier.Source: §3.5.2 · p. 21
• A restart-tier event rolls training back to the most recent fully committed checkpoint, after which the job is relaunched and notifications are sent.The rollback uses a fixed checkpoint interval; the monitoring system detects anomalies and backs up relevant checkpoints.Source: §3.5.2 · pp. 20–21

3.5.3 Training Dynamics & Stability

• Most loss spikes occur in the first 40% of pre-training, especially the first 20%, and early instability can be exacerbated by poorly tuned τepoch.Figure 10 shows the concentration across training.Source: §3.5.3 · p. 22, Figure 10 · p. 23
• Rollbacks target wide, malignant spikes rather than narrow spikes; gradient norms peak mid-training and decay, while parameter norms peak near step 200k and drift downward.Gradient-norm spikes are more frequent in the first half; the observed late gradient-norm decay is not standard behavior and warrants further investigation.Source: §3.5.3 · p. 23, Figure 11 · p. 23, Figure 12 · p. 24

4.1 Mid-training Recipe

• Four mid-training stages progressively expand context length to 64K, 128K, and 512K after an initial 8K stage, while retaining at least 30% short-context data.The RoPE base frequencies for the expanded stages are 1M, 10M, and 10M; Figure 14 shows the context-length bucket mixes.Source: §4.1 · pp. 24–25, Figure 14 · p. 28

4.2 Mid-training Data: TxT360-Midas

• TxT360-Midas combines synthetic and natural text to extend context, introduce reasoning behaviors, and compensate for web-data limitations, including with over 250 million mathematical problems.The four-stage curriculum organizes data by token length.Source: §4.2 · p. 25

4.2.1 LLM Thinking Traces

• The thinking-traces corpus uses over 250 million unique mathematical problems, with solutions generated by Qwen3-32B and GPT-OSS-120B, and adds coding and STEM traces from Nemotron data.Only train subsets are used where relevant to avoid contamination.Source: §4.2.1 · pp. 25–26

4.2.2 Synthetic Reasoning Behaviors

• Synthetic reasoning behaviors use natural user queries and templates for over 100 behaviors to encourage behavioral diversity and prepare for later reasoning improvements such as RLVR-style training.Other synthetic behaviors include science discussion, planning, and user manuals.Source: §4.2.2 · p. 26

4.2.3 Data mixes

• The four-stage mixes combine web, math, code, and institutional-book data, organized into four context-length buckets; K2’s AIME memorization is reported as low relative to other models.The authors interpret K2’s strong AIME performance as driven by reasoning ability rather than memorization alone.Source: §4.2.3 · pp. 27, 29, Figure 13 · p. 28, Figure 14 · p. 28
• K2’s AIME-2025 pass rate rises from 0 before mid-training to over 80% pass@64, with non-trivial pass@1; stage 4 shows mild degradation.Stage 4 emphasizes long-context data above 128k, where high-quality reasoning data is difficult to obtain.Source: §4.2.3 · p. 28, Figure 15 · p. 29

4.3 Mid-training Infrastructure

• The in-house mid-training framework supports TP, CP, and FSDP; a feature enabled from stage 2 significantly reduces unnecessary document truncations with only a few additional padding tokens and improves loss on the same model and data.CP is 1 in stages 1–2, 2 in stage 3, and 8 in stage 4.Source: §4.3 · p. 29

4.4 The Mid-training Run

• Across mid-training, traditional benchmarks generally remain stable or improve slightly while complex reasoning benchmarks gain; GPQA-Diamond improves across the final three stages, and a 10M RoPE base restores 128K performance after 1M degraded it.The authors attribute unstable GSM8K results to output-format parsing errors and stage-specific mathematical data distributions.Source: §4.4 · p. 30

5.1 Evaluation Methodology

• The evaluation covers likelihood-based benchmarks and generative tasks across math, STEM, coding, instruction following, and logic, using 19 benchmarks and comparable-scale open-model baselines.The comparison uses a common evaluation pipeline; GSM8K and MATH report the best results across few-shot CoT and reasoning prompts.Source: §5 · p. 30, §5.1 · pp. 32–33

5.2 Evaluation Results

• Across pretraining and four mid-training stages, K2 improves broadly, with the strongest gains in mathematics, STEM reasoning, and logic.The mid-training models substantially outperform comparable-scale open-source baselines on these reasoning-intensive tasks.Source: §5.2 · pp. 33–34
• The mid-4 model leads compared models on GPQA-Diamond (55.1), GSM8K (93.6), and MATH (94.7).K2 does not match Qwen2.5-72B on overall MMLU (86.1), while mid-3 leads open models on MMLU-Pro (59.8).Source: §5.2 · p. 33
• K2 improves steadily on coding tasks, but Qwen2.5-72B remains strongest on MBPP (75.4); mid-4 scores 61.8 on MBPP and 54.3 on HumanEval pass@1.MBPP remains a relative weakness for K2.Source: §5.2 · p. 34
• Mid-4 reaches 75.6 on Countdown, 92.9 on KK-4 People, 82.8 on KK-8 People, and 87.6 on Order-15 Items.Its KK-8 People result matches the reported 83% for DeepSeek-R1 and o3-mini-high.Source: §5.2 · p. 34

5.2.1 Evaluation with Reasoning

• Reasoning evaluation requires careful prompt, response-length, and parsing choices, and can require token budgets up to 32K, increasing inference cost.Answer-format mismatches can change benchmark scores when parsers do not align.Source: §5.2.1 · p. 34
• On GSM8K, structured reasoning prompts with expanded token budgets and precise parsing yield the strongest reported result: 93.6 at mid-3.Scores rise from 68.0 at base to 83.8 at mid-4 in standard QA; CoT reaches 90.0.Source: §5.2.1 · p. 35

5.2.2 Long Context

• Mid-4 remains stable through 128K context, with NIAH accuracy above 95% there, although its short-context RULER scores trail leading baselines slightly.The authors attribute limits at extreme context lengths to scarce naturally occurring long documents.Source: §5.2.2 · p. 36

6 Supervised Fine-tuning

• A single SFT model supports low, medium, and high reasoning efforts through chat-template settings, offering varying test-time compute budgets instead of separate instruct and thinking models.The authors report that high effort has a wider generation-length spread, suggesting potential to adapt generation length to problem difficulty.Source: §6 · pp. 36–37, §6.1 · p. 37

6.1 SFT Recipe

• Across challenging math and science benchmarks, performance increases with reasoning effort and longer generations; high effort shows a wider generation-length spread than low and medium.The figure compares AIME 2025, HMMT 2025, and GPQA-Diamond across generation-length quartiles.Source: §6.1 · p. 37, Figure 19 · p. 37

6.2.1 Core SFT data

• Core SFT data combines public or synthetic question-answer pairs, regenerated answers, and curated material spanning math, code, chat, STEM, self-identity, safety, and constrained instruction following.The core question-answer data is quality-filtered, deduplicated, and decontaminated against evaluation benchmarks.Source: §6.2.1 · p. 38
• Self-identity data uses generated queries and responses, while safety data combines harmful-intent seeds with adversarial and jailbreak variants.The self-identity collection includes about 3,000 English queries and translated samples across 10 target languages.Source: §6.2.1 · p. 39

6.2.2 Agentic and Tool-use Data

• To address scarce multi-turn agentic data, the authors build synthetic pipelines, including teacher-student math dialogues and simulated tool-use interactions.The math dialogues train extended solution discussions and critique; tool-use data is standardized and filtered for interaction consistency.Source: §6.2.2 · pp. 39–40

6.2.3 SFT data mix

• The SFT mixture contains about 10 million documents and 10B loss tokens; approximately 98% of its data is under 8K tokens, without explicit length-distribution control.Self-identity prompts are included with different probabilities by data category rather than in every conversation.Source: §6.2.3 · pp. 40–41

6.3 SFT Training

• Full-parameter SFT trains the final mid-training checkpoint on the conversation data using a chat template and best-fit sequence packing.Packing avoids truncation by skipping samples longer than the sequence length.Source: §6.3 · p. 41
• Packing yields zero average truncation and padding below 0.004%, compared with more than 95% padding under naive batching.The run spans three epochs, or 2,100 steps.Source: §6.3 · p. 41

7.1.2 Evaluation Results

• After simple SFT, K2 is competitive across many domains and comparable to fully trained reasoning and instruct models of larger scales.The table compares K2 SFT models with open-weight and open-source models.Source: §7.1.2 · p. 43, Table 9 · p. 44

Long Context.

• K2-High scores 42.6% accuracy on LongBench V2, outperforming all reasoning/thinking models except GPT-OSS but slightly underperforming Qwen 2.5 72B and DeepSeek V3.1-Instruct.Source: Long Context · p. 43
• Across math benchmarks, higher reasoning effort yields larger gains on olympiad-level AIME and HMMT than on easier GSM8K or average-difficulty MATH.On function calling, medium reasoning effort performs best overall for both K2 and GPT-OSS.Source: Long Context · pp. 43, 45
• K2-High achieves 77.0 on MMLU-PRO and outperforms several instruct models and large MoE baselines, while remaining behind leading large MoE thinking models on math and STEM.K2 also outperforms several larger MoEs on AIME and HMMT; the supplied passages do not give those scores.Source: Long Context · pp. 43–44
• K2-High scores 62.1 on Arena Hard V2 and 90.0 on IFEval, while K2-High outperforms instruct models on three coding benchmarks but trails several reasoning models.K2 remains competitive with larger reasoning-oriented models despite not yet using reinforcement learning or preference optimization.Source: Long Context · pp. 44–45
• K2-Medium scores 52.38 overall on BFCL-v4, above Qwen 2.5-72B (45.88) and Llama 3.3-70B (31.05); K2-High reaches 93.50% on multiple calls and 89.00% on parallel calls.K2-Medium also scores 50.62 on multi-turn interactions, versus 33.88 for Qwen 2.5 and 19.62 for Llama 3.3.Source: Long Context · p. 45, Table 10 · p. 46

7.2 Assessing K2’s Reasoning Potential

• K2-SFT’s pass rates improve as group size grows across math, code, and instruction-following benchmarks, with the largest gains at high reasoning effort.The pass@k analyses use at most 32k tokens during inference and suggest headroom for further reinforcement learning.Source: §7.2 · p. 46, Figure 22 · p. 47
• For K2 rollout sampling, performance improves stably as temperature approaches 1.0, shows slight degradation at 1.2, and deteriorates sharply thereafter, identifying a stable region and temperature limit for continued post-training.Source: §7.2 · pp. 46–47

8 A Longitudinal Capability Study

• Across 10 checkpoints spanning five training stages, capability scores shift from low in early training to high in later training, with three phases of development identified.The checkpoints track the pre-training baseline and key mid-training intervals, including context extensions from 8k to 512k.Source: §8 · pp. 48–49, Figure 24 · p. 49

8.1 LLM-Based “Vibe”-Check

• Claude scores each checkpoint response from 0.0 to 1.0 across six dimensions, and aggregate scores trace a shift from basic text generation to complex multi-step reasoning.The dimensions include correctness, reasoning quality, creativity, coherence, completeness, and task-specific adherence.Source: §8.1 · pp. 49–50

8.2 Qualitative Analysis

• Qualitative comparisons show a shift from repetition to multi-step deduction, from abstract plans to actionable Stage 4 plans, and from generalist explainability toward degradation after math specialization.The last comparison documents a trade-off associated with domain optimization.Source: §8.2 · p. 50, Figure 25 · p. 50, Figure 26 · p. 51

8.3 Additional Demonstrations of Capability Shifts

• Additional examples show later-stage gains in Fermi estimation, system design, and handling ambiguity compared with early failures or looping.The examples depict the final model decomposing an estimation task, proposing a modular architecture, and giving a placeholder-based solution to an ambiguous prompt.Source: §8.3 · p. 52, Figure 28 · p. 53, Figure 29 · p. 53

• The paper categorizes existing LLMs as closed-source, open-weight, or 360-open-source.It presents the categories as part of its overview of notable models.Source: §9 · p. 54
• Closed-source models are generally offered as services through APIs or chatbots, with some also providing fine-tuning services or embeddings.Examples include the GPT, Claude, and Gemini families.Source: §9 · p. 54
• Many open-weight releases provide final weights and architecture but keep data sources and most training details undisclosed.The paper lists Llama, Mistral, Gemma, Qwen, GLM, and other model families as examples.Source: §9 · pp. 54–55
• The 360-open-source category is described as fully open and reproducible, with both data and code available.Examples include Pythia, BLOOM, GPT-NeoX, OLMo, and models in the LLM360 ecosystem.Source: §9 · p. 55

10 Safety Evaluation and Red Teaming

• K2 produces 12,702 desired responses across 72 datasets and 14,700 samples, an overall safety score of 86.41%.The evaluations uniformly sample 200 instances per benchmark.Source: §10 · pp. 55–56
• Under the same protocol, K2 consistently outperforms K2-Think in safety coverage and refusal robustness across assessed categories.Source: §10 · p. 56, Figure 32 · p. 56
• Most of 16 risk categories exceed 95% safety; selected knowledge- and regulation-intensive domains exceed 97%, while socially sensitive categories score 94–97%.Privacy scores 89.8%, and a small number of narrow physical-safety subtypes are also lower.Source: §10 · p. 56, Figure 33 · p. 57

10.1 Identified Safety Issues

• K2 scores 0.11 on Or-Bench and 0.41 on Do-Not-Answer FP, where harmless prompts phrased like unsafe requests expose over-refusal.Manual inspection attributes some refusals to sensitivity to superficial linguistic cues.Source: §10.1 · p. 57
• More than half of low-scoring datasets involve jailbreak attacks, and even the strongest checkpoints can be circumvented by sufficiently adversarial prompts.The authors note that jailbreak strategies are diverse and evolving despite dedicated resistance data in training.Source: §10.1 · p. 57
• K2 is not trained to resist prompt extraction, reducing its suitability for settings where protecting internal instructions is necessary.The authors connect this choice to their transparent, fully open-source design.Source: §10.1 · p. 57

10.2 Safety of Thinking Traces

• The Thinking–Response Divergence metric quantifies discrepancies between reasoning traces and surfaced responses, and traces can contain undesired information even when final responses are correct.The authors describe cases where reasoning reveals a password embedded in a system prompt.Source: §10.2 · p. 58, Figure 34 · p. 58

B List of Model Compared

• Table 11 lists the models used in the experiments and their HuggingFace repository locations.Source: Table 11 · p. 60