PaperlayerView exact revision on arXiv

Source-linked AI summary

K-EXAONE Technical Report

Eunbi Choi, Kibong Choi, Seokhee Hong, Junwon Hwang, Hyojin Jeon, Hyunjik Jo, Joonkee Kim, Seonghwan Kim, Soyeon Kim, Sunkyoung Kim, Yireun Kim, Yongil Kim, Haeju Lee, Jinsik Lee, Kyungmin Lee, Sangha Park, Heuiyeen Yeen, Hwan Chang, Stanley Jungkyu Choi, Yejin Choi, Jiwon Ham, Kijeong Jeon, Geunyeong Jeong, Gerrard Jeongwon Jo, Yonghwan Jo, Jiyeon Jung, Naeun Kang, Dohoon Kim, Euisoon Kim, Hayeon Kim, Hyosang Kim, Hyunseo Kim, Jieun Kim, Minu Kim, Myoungshin Kim, Unsol Kim, Youchul Kim, YoungJin Kim, Chaeeun Lee, Chaeyoon Lee, Changhun Lee, Dahm Lee, Edward Hwayoung Lee, Honglak Lee, Jinsang Lee, Jiyoung Lee, Sangeun Lee, Seungwon Lim, Solji Lim, Woohyung Lim, Chanwoo Moon, Jaewoo Park, Jinho Park, Yongmin Park, Hyerin Seo, Wooseok Seo, Yongwoo Song, Sejong Yang, Sihoon Yang, Chang En Yea, Sihyuk Yi, Chansik Yoon, Dongkeun Yoon, Sangyeon Yoon, Hyeongu Yun

arXiv:2601.01739v2cs.CLcs.AI

TL;DR

K-EXAONE addresses the challenge of developing globally competitive large models amid Korea’s AI infrastructure constraints. It combines a sparse MoE architecture with hybrid attention and multilingual coverage; evaluations report strengths across reasoning, agentic abilities, general knowledge, multilingual understanding, and long-context processing.

  • Problem

    Korea faces relative shortages of AI-specialized data centers and chips, while developing a globally top-tier foundation model remains a stated need amid competition and large-scale model growth.

    Source: §1 · p. 2

  • Method

    K-EXAONE combines sparse MoE routing, hybrid attention, and an expanded multilingual architecture, with approximately 23B of its 236B parameters activated.

    Source: §2.1 · p. 2, §1 · p. 2

  • Results

    Evaluations report strengths in reasoning, agentic capabilities, general knowledge, multilingual understanding, and long-context processing.

    Source: §7 · p. 10

  • Takeaways & Limitations

    Support for Korean, English, Spanish, German, Japanese, and Vietnamese makes K-EXAONE versatile across diverse linguistic contexts.

    Source: §7 · p. 10

    Responses may contain harmful or biased content, be semantically or syntactically incorrect, or be false or contradictory because they do not reflect the latest information.

    Source: §5 · p. 10

Abstract

from arXiv · show

This technical report presents K-EXAONE, a large-scale multilingual language model developed by LG AI Research. K-EXAONE is built on a Mixture-of-Experts architecture with 236B total parameters, activating 23B parameters during inference. It supports a 256K-token context window and covers six languages: Korean, English, Spanish, German, Japanese, and Vietnamese. We evaluate K-EXAONE on a comprehensive benchmark suite spanning reasoning, agentic, general, Korean, and multilingual abilities. Across these evaluations, K-EXAONE demonstrates performance comparable to open-weight models of similar size. K-EXAONE, designed to advance AI for a better life, is positioned as a powerful proprietary AI foundation model for a wide range of industrial and research applications.

1 Introduction

K-EXAONE combines sparse MoE scaling and long-context attention with expanded multilingual coverage; its redesigned tokenizer improves average efficiency across language and technical inputs.

1 Introduction

  • Against a global scaling race and Korea’s compute-resource constraints, K-EXAONE builds on EXAONE 4.0’s reasoning/non-reasoning hybrid and adds German, Japanese, and Vietnamese coverage.EXAONE 4.0 supports Korean, English, and Spanish.Source: §1 · p. 2

2.1 Model Configurations

  • K-EXAONE uses 128-expert sparse MoE routing, activating eight routed experts plus one shared expert per token; 236B total parameters yield approximately 23B active.The first layer is dense for training stability.Source: §2.1 · p. 2, Figure 2 · p. 3
  • K-EXAONE’s MTP module supervises an additional future token during training and enables self-drafting at inference, improving decoding throughput by approximately 1.5× over standard autoregressive decoding.Source: §2.1 · p. 2, Figure 2 · p. 3
  • Hybrid attention supports 256K-token contexts with lower memory and compute than global attention across all layers.QK Norm stabilizes training; RoPE is applied only in SWA layers, and the SWA window drops from 4,096 to 128 to reduce KV-cache use.Source: §2.1 · pp. 2–3

2.2 Tokenizer

  • K-EXAONE’s tokenizer improves bytes-per-token efficiency by approximately 30% on average over the previous EXAONE tokenizer.The reported gains span English, Korean, multilingual, STEM, and code inputs.Source: §2.2 · p. 4

3 Training

K-EXAONE training combines multilingual and reasoning-focused pre-training with staged context extension and three post-training stages. Its procedures include verifiable-reward reinforcement learning, preference learning, and data-compliance reviews.

3.1 Pre-training

  • Pre-training uses a three-stage curriculum, filtered data, expanded German, Japanese, and Vietnamese coverage with targeted synthesis, and document-grounded thinking trajectories.The synthesized multilingual corpora aim to balance knowledge across languages, while thinking-augmented samples encode step-by-step inference.Source: §3.1 · p. 4

3.2 Context Length Extension

  • Context extension proceeds from 8K to 32K to 256K tokens, mixing rehearsal, synthetic reasoning, and end-to-end long-document data; short-context checks and NIAH tests guide each stage.Rehearsal is intended to preserve short-context performance, and training continues until NIAH performance is consistently near-perfect across target ranges.Source: §3.2 · p. 5

3.3 Post-training

  • Post-training comprises supervised fine-tuning, reinforcement learning on reasoning-intensive and verifiable tasks, and preference learning; it also uses synthetic tool tasks and web-search sub-agents.Generated tool environments are filtered for unrealistic or unsolvable cases; web-search sub-agents summarize pages and compress interaction histories.Source: §3.3 · pp. 5–6

3.4 Reinforcement Learning

  • Reinforcement learning covers math, code, STEM, and instruction-following, using rule-based and LLM-judge verification with AGAPO and truncated importance sampling.The training procedure drops prompts whose rollouts all receive identical rewards, combines group-level advantages with global normalization, omits the KL penalty, and freezes the MoE router.Source: §3.4 · p. 6

3.5 Preference Learning

  • GROUPER preference learning samples multiple responses per query, combines rule-based and rubric-based rewards, and uses standardized, scaled group-level advantages in a SimPER-style objective.The stage focuses on general alignment domains while aiming to preserve reasoning performance; the report says GROUPER improves general-domain performance.Source: §3.5 · p. 6

3.6 Data Compliance

  • AI Compliance reviews address legal risks across data collection, model training, and information provision.The report identifies copyright, intellectual-property, and personal-information protection issues as examples of those risks.Source: §3.6 · p. 7

4 Evaluation

K-EXAONE is evaluated across reasoning, agentic, general, Korean, multilingual, and safety benchmarks. The reported results are competitive across these areas, with task-specific strengths in mathematics, coding, and long-context understanding.

4.1 Benchmarks and Setup

  • K-EXAONE is evaluated on a diverse benchmark suite spanning nine categories, including world knowledge, math, coding/agentic coding, instruction following, and long-context understanding benchmarks such as AA-LCR and OPENAI-MRCR.Source: §4.1 · p. 7

4.2 Results

  • In agentic evaluations, K-EXAONE scores 29.0 on TERMINAL-BENCH 2.0, 49.4 on SWE-BENCH VERIFIED, and 73.2 weighted average on τ2-BENCH.The benchmarks assess multi-step interaction and tool use, including agentic coding and tool selection.Source: §4.2 · p. 7
  • K-EXAONE performs competitively and often leads across evaluated reasoning tasks, surpassing gpt-oss-120b and Qwen3-235B-A22B-Thinking-2507 on nearly all listed mathematics benchmarks.The exception is HMMT NOV 2025 against Qwen3-235B-A22B-Thinking-2507; it also outperforms most compared baselines on LIVECODEBENCH V6 and is comparable on LIVECODEBENCH PRO.Source: §4.2 · p. 7, Table 3 · p. 8
  • In NON-REASONING mode, K-EXAONE scores 45.2 on AA-LCR and 65.9 on OPENAI-MRCR, surpassing strong baselines by a large margin.The report characterizes these results as efficient, accurate long-context processing.Source: Table 4 · p. 9, §4.2 · p. 9
  • In REASONING mode, K-EXAONE scores 67.3 on IFBENCH and 89.7 on IFEVAL, outperforming most competing models on instruction following.The report also describes its REASONING-mode long-context results as competitive: 53.5 on AA-LCR and 52.3 on OPENAI-MRCR.Source: §4.2 · p. 9
  • K-EXAONE scores 67.3 on KMMLU-PRO, 61.8 on KOBALT, 83.9 on CLICK, 90.9 on HRM8K, and 86.8 on KO-LONGBENCH.The report describes these as competitive results in Korean professional knowledge, language competence, mathematical reasoning, and long-context capability.Source: §4.2 · pp. 9–10
  • Across the five evaluated non-English supported languages, K-EXAONE scores 85.7 on MMMLU and averages 90.5 on WMT24++.The report describes its multilingual knowledge understanding as competitive and its translation quality as stable.Source: §4.2 · pp. 7, 10
  • K-EXAONE achieves competitive performance on WILDJAILBREAK and KGC-SAFETY, assessing harmful-prompt robustness and Korean sociocultural and global ethical contexts.The authors report that this is achieved without significant performance trade-offs on general downstream tasks.Source: §4.2 · p. 10

5 Limitations

K-EXAONE may generate inappropriate, biased, or incorrect responses. These risks are associated with content in training data and the model's reliance on training-data statistics.

5 Limitations

  • Responses may contain personal, harmful, or otherwise inappropriate information, and may reflect biases associated with age, gender, or race.The report notes that efforts to exclude harmful and biased information from training data cannot ensure that all problematic content is removed.Source: §5 · p. 10
  • Because responses rely heavily on training-data statistics and do not reflect the latest information, they may be linguistically incorrect, false, or contradictory.Source: §5 · p. 10

6 Deployment

K-EXAONE combines a Mixture-of-Experts architecture, hybrid attention, multilingual coverage, and a 256K-token context window. The report presents benchmark performance across several capability areas and points users to license terms for model use.

6 Deployment

  • The report directs users to Appendix B for license information, which it describes as essential for legal utilization of the model.Source: §6 · p. 10

7 Conclusion

  • K-EXAONE uses MoE scaling and hybrid attention to handle long-context inputs and outputs, and supports Korean, English, Spanish, German, Japanese, and Vietnamese.Source: §7 · p. 10
  • The model supports a maximum context length of 256K tokens and undergoes SFT, reinforcement learning on verifiable rewards, and preference learning after pretraining.Pretraining uses a three-stage curriculum and FP8 precision training.Source: §7 · p. 10
  • Benchmark evaluations report strength in reasoning, agentic capabilities, general knowledge, multilingual understanding, and long-context processing.Source: §7 · p. 10

A Contributors

The appendix lists contributors and defines terms used in the K-EXAONE AI Model License Agreement. The agreement grants a broad license subject to conditions on distribution and sublicensing.

A Contributors

  • The report says all authors are listed alphabetically by last name and names core contributors and contributors separately.Source: A Contributors · p. 11

1. Definitions

  • The license defines Model, Derivative Work, Output, Licensor, and User, including modifications and materials generated by the model.Source: §1 · p. 12

2. License Grant

  • The licensor grants users a non-exclusive, non-transferable, worldwide, irrevocable license to access, modify, use, distribute, and create derivative works for commercial or non-commercial purposes.Distributed copies must include the agreement and use names beginning with “K-EXAONE”.Source: §2 · p. 12
  • Commercial distribution, sublicensing, or making the model or derivative works available to third parties requires a separate agreement with the licensor.Source: §2 · p. 12

3. Exceptions and Restrictions

The terms restrict reverse engineering, illegal and unethical use, allocate ownership, and disclaim warranties for the model and its outputs.

3. Exceptions and Restrictions

  • Users may not reverse engineer the model or use it for illegal, fraudulent, or unauthorized activities.The agreement also prohibits circumventing technical protection measures.Source: §3 · p. 12
  • Ethical-use rules prohibit harmful or misleading content, discrimination, rights violations, and uses that cause harm.Users must obtain necessary permissions for uses affecting third-party rights and take reasonable measures to prevent harm.Source: §3 · p. 13

4. Ownership

  • The agreement transfers no ownership or patent rights in the model to users, while generated output remains the user's responsibility.The licensor claims no ownership over output.Source: §4 · p. 13

5. Warranty

  • The licensor provides the model “as is,” disclaims warranties and guarantees about performance, and leaves users responsible for assessing output accuracy and suitability.The agreement says the model may contain bugs, errors, or other limitations.Source: §5 · p. 13

6. Limitation of Liability

The agreement limits the licensor's liability, specifies termination and dispute terms, and requires users to accept the agreement when using the model. The evaluation setup details say internal evaluations generally follow each benchmark's official setup.

6. Limitation of Liability

  • The licensor disclaims liability for specified damages, while users agree to indemnify the licensor and its affiliates for claims connected to use or breach.The damages list includes business interruption and data loss.Source: §6 · p. 13

7. Termination

  • After termination, users must stop using and destroy all model copies, while certain agreement provisions survive termination.The licensor may terminate for breach, with or without cause and without prior notice.Source: §7 · p. 14

8. Governing Law

  • The agreement applies Republic of Korea law and assigns disputes to KCAB arbitration in Seoul, conducted in Korean by one arbitrator.Arbitration follows the KCAB International Arbitration Rules in effect when proceedings commence.Source: §8 · p. 14

9. Miscellaneous

  • The agreement supersedes prior understandings, rejects inconsistent user purchase-order terms, and binds users who download, install, or use the model.Users acknowledge having read and understood the agreement.Source: §9 · p. 14

C Evaluation Setup Details

  • Internal evaluation generally follows each benchmark's official setup; the report supplies specific settings for benchmarks evaluated in its internal environment.Benchmarks not otherwise mentioned use the official setup.Source: C Evaluation Setup Details · p. 15

C.1 Multiple-Choice Questions

The evaluation appendix describes prompt and scoring procedures for multiple-choice and math benchmarks, along with agent and judge choices for two other benchmarks.

C.1 Multiple-Choice Questions

  • For multiple-choice benchmarks, prompts in Figures 4–5 request a final option letter, which is parsed for scoring.The English template also instructs models to think step by step.Source: §C.1 · p. 15, Figure 4 · p. 15, Figure 5 · p. 15

C.2 Evaluation Prompts

  • Math benchmark answers are parsed and checked against ground truth by exact match or LLM-based equality, except IMO-ANSWERBENCH uses official evaluation and judging prompts.The math prompts are shown in Figures 6–7.Source: §C.2 · p. 15, Figure 6 · p. 16, Figure 7 · p. 16

C.3 HUMANITY’S LAST EXAM

  • For HUMANITY’S LAST EXAM, internal evaluation uses gpt-5-mini-2025-08-07 as the judge model with the official judge prompt.Source: §C.3 · p. 16

C.4 TERMINAL-BENCH 2.0

  • TERMINAL-BENCH 2.0 evaluations use Terminus 2 as the default agent when official scores are unavailable.Source: §C.4 · p. 16

C.5 SWE-BENCH VERIFIED

The appendix specifies agents, scoring protocols, and judge settings for several benchmarks, including reasons for not reproducing some baseline scores.

C.5 SWE-BENCH VERIFIED

  • SWE-BENCH VERIFIED uses mini-SWE-agent as the default agent and the same setup to obtain scores when official results are unavailable.Source: §C.5 · p. 16

C.6 BROWSECOMP

  • BROWSECOMP uses a summarizer and trajectory compressor, with up to 500 tool-call steps and compression every 50; baseline scores are not reproduced because agent pipelines vary.Reported baseline scores are included only when available from official technical reports or blogs.Source: §C.6 · p. 16

C.7 IFBENCH and IFEVAL

  • IFBENCH uses the prompt-loose metric, while IFEVAL uses prompt-strict.Source: §C.7 · p. 16

C.8 OPENAI-MRCR

  • OPENAI-MRCR scores average 2-, 4-, and 8-needle results within bins up to 128K tokens, then macro-average the bin scores.The protocol requires prepending the provided alphanumeric hash and computes scores with difflib.SequenceMatcher.Source: §C.8 · p. 16

C.9 WMT24++

  • WMT24++ uses the official judge prompt with gpt-5-mini-2025-08-07, averaging LLM-judge translation scores across English and five supported non-English languages.Figure 8 presents the judging prompt.Source: §C.9 · p. 17, Figure 8 · p. 17

C.10 WILDJAILBREAK

  • WILDJAILBREAK safety is evaluated by Qwen3Guard-Gen-8B, reporting the proportion of test cases classified as safe.Higher Safe Rate indicates a safer model.Source: §C.10 · p. 17

D.1 Code Utility Benchmark (CODEUTILITYBENCH)

CODEUTILITYBENCH evaluates practical coding workflows, while the surrounding sections describe Korean long-context and safety benchmarks and report balanced multilingual performance gains. K-EXAONE improves over its predecessor on CODEUTILITYBENCH, with room to improve in Maintenance.

D.1 Code Utility Benchmark (CODEUTILITYBENCH)

  • CODEUTILITYBENCH addresses the limited coverage of competitive-programming benchmarks with 16 tasks across Understanding, Implementation, Refinement, and Maintenance.Its 300 test instances cover 15 tasks with 20 instances each, excluding visualize; experts reviewed the queries and rubrics.Source: §D.1 · p. 18
  • 71.9% overall on CODEUTILITYBENCH exceeds EXAONE-4.0-32B’s 63.2%, with marked gains in Understanding and Implementation but room to improve in Maintenance.The benchmark evaluates practical real-world coding workflows.Source: Figure 9 · p. 18, §D.1 · p. 18

D.2 KO-LONGBENCH

  • KO-LONGBENCH assesses Korean long-context understanding across document QA, story and dialogue-history understanding, in-context learning, structured QA, and RAG.Source: §D.2 · p. 19

D.3 KGC-SAFETY

  • K-AUT combines universal ethical principles with Korean sociocultural context, and KGC-SAFETY evaluates the resulting standards.Source: §D.3 · p. 19

E.1 Multilingual

F Safety

The safety section introduces K-AUT, which combines universal ethical principles with Korean sociocultural context, and KGC-SAFETY, a benchmark for evaluating adherence to those standards. Models tend to score lower on Korean Sensitivity and Future Risk than on Universal Human Values and Social Safety.

F.1 Korea-Augmented Universal Taxonomy (K-AUT)

  • K-AUT augments universal human values with regional specificities across four domains and 226 risk areas; its Korean Sensitivity domain covers local issues such as historical and geopolitical conflicts.The framework also draws on international consensus, domestic laws, verified historical records, and predictive ethics.Source: §F.1 · p. 21, Table 7 · p. 21

F.2 KGC-SAFETY : Korean Global Civic Safety Benchmark

  • KGC-SAFETY contains 2,260 test instances, sampling 10 cases for each K-AUT category across six languages and multi-turn, adversarial, and naive settings.Evaluation uses an LLM-as-a-judge framework and reports Safe Rate as the proportion of cases classified as safe.Source: §F.2 · p. 22, Table 8 · p. 22
  • Most models have relatively high Safe Rates on Universal Human Values and Social Safety, but lower rates on Korean Sensitivity and Future Risk.Source: Table 9 · p. 22, §F.2 · p. 22
Loading 2601.01739v2…