DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

1. Introduction

• The introduction says proprietary-model performance has accelerated faster than open-model performance, widening the gap on complex tasks.Source: §1 · p. 2
• The identified bottlenecks are vanilla-attention inefficiency, too little post-training compute, and weaker agent generalization and instruction following than proprietary counterparts (Li et al., 2025; Luo et al., 2025).Source: §1 · p. 2
• The proposed response combines DSA, scalable RL, and agentic task synthesis to address efficiency, reasoning, and tool-use generalization.DSA preserves performance in long-context scenarios; RL uses more than 10% of pre-training compute, while a DeepSeek-V3 (DeepSeek-AI, 2024) cold start precedes synthesis of over 1,800 environments and 85,000 prompts.Source: §1 · p. 2
• DeepSeek-V3.2 performs similarly to Kimi-k2-thinking and GPT-5 on reasoning benchmarks, while Speciale matches Gemini-3.0-Pro and shows gold-medal performance in four 2025 competitions.DeepSeek-V3.2 also advances open-model agent capabilities on long-tail tasks and is presented as a cost-efficient alternative that narrows the gap with frontier proprietary models.Source: §1 · p. 2

2.1. DeepSeek Sparse Attention

• Compared with DeepSeek-V3.1-Terminus, DSA is the sole architectural modification, using a lightning indexer to select top-k key-value entries for sparse attention.The indexer scores preceding tokens relative to each query; DSA is instantiated under MLA in MQA mode, sharing each latent key-value vector across that query token’s heads.Source: §2.1 · p. 3

2.1.1. Continued Pre-Training

• Continued pre-training uses two stages on data aligned with DeepSeek-V3.1-Terminus’s 128K extension: dense indexer warm-up followed by sparse training that adapts all model parameters.Warm-up freezes the model except the indexer and aligns it to normalized main-attention scores; sparse training uses selected tokens, detaches indexer inputs, and trains the indexer and main model with separate objectives.Source: §2.1.1 · pp. 3–4

2.2. Parity Evaluation

• Benchmarks showed similar performance to DeepSeek-V3.1-Terminus; Elo scores were closely matched, while AA-LCR reasoning scores were four points higher and Fiction.liveBench results consistently favored DeepSeek-V3.2-Exp.The benchmark comparison covered short- and long-context tasks and reported no substantial performance degradation.Source: §2.2 · p. 5

2.3. Inference Costs

• DSA reduces the main model’s core attention complexity and achieves significant end-to-end speedup in long-context scenarios; masked MHA simulates DSA for short-sequence prefilling.The selected-token count k is much smaller than sequence length L; the short-context implementation is used to achieve higher efficiency under short-context conditions.Source: §2.3 · p. 5

3. Post-Training

• Specialists cover six domains in thinking and non-thinking modes, and their distilled data’s small performance gap is effectively eliminated by subsequent RL.The same post-training pipeline as DeepSeek-V3.2-Exp uses sparse attention, specialist distillation, and mixed RL; separate models generate long-chain-of-thought and direct-response data.Source: §3 · pp. 5–6
• For mixed RL, GRPO combines reasoning, agent, and human-alignment training in one stage to balance domains and avoid multi-stage catastrophic forgetting.Reasoning and agent tasks use rule-based outcome reward, length penalty, and language consistency reward; general tasks use prompt-specific generative-reward rubrics.Source: §3 · p. 6
• DeepSeek-V3.2 combines specialist-distilled reasoning, agent, and alignment data with thousands of continued RL steps; Speciale instead trains only on reasoning data with a reduced length penalty.Speciale also uses the dataset and reward method from DeepSeekMath-V2 to strengthen mathematical proofs.Source: §3 · p. 6

3.1. Scaling GRPO

• An importance-sampling correction makes the KL estimate’s gradient unbiased, while KL strength can vary by domain and may be weak or omitted for mathematics.The correction addresses noisy updates from the original K3 estimator when sampled tokens have substantially lower probability under the current policy than the reference policy.Source: §3.1 · p. 7
• Off-Policy Sequence Masking excludes negative-advantage sequences with substantial policy divergence, improving stability in scenarios that otherwise become unstable.The method addresses off-policy effects from rollout batching and training-inference inconsistencies.Source: §3.1 · pp. 7–8
• Keep Routing preserves sampled MoE expert paths during training, while Keep Sampling Mask applies the sampling-time top-p or top-k truncation to the training policy.The authors report Keep Routing is crucial for MoE RL stability and that top-p with Keep Sampling Mask preserves language consistency.Source: §3.1 · p. 8

3.2.1. Thinking Context Management

• Reasoning is retained across tool-only messages and discarded only when a new user message arrives, avoiding redundant re-reasoning while preserving tool-call history.Frameworks that simulate tool interactions as user messages may not benefit fully; the authors recommend non-thinking models for those architectures.Source: §3.2.1 · p. 9

3.2.2. Cold-Start

• Cold-start prompting combines reasoning and non-reasoning agentic data so the model can sometimes produce tool-use reasoning trajectories for subsequent RL.The approach relies on the model following explicit instructions to incorporate tool execution into its reasoning.Source: §3.2.2 · pp. 9–10

4.1. Main Results

• DeepSeek-V3.2 performs similarly to GPT-5-high on reasoning tasks but slightly below Gemini-3.0-Pro; it matches K2-Thinking with substantially fewer output tokens.The comparison with K2-Thinking is reported in Table 3.Source: §4.1 · p. 13, Table 2 · p. 13, Table 3 · p. 15
• In code-agent evaluations, DeepSeek-V3.2 significantly outperforms open-source models on SWE-bench Verified and Terminal Bench 2.0.Terminal Bench 2.0 scored 46.4 using Claude Code; non-thinking mode with Terminus scored 39.3. SWE-bench Verified robustness tests across other frameworks and non-thinking mode ranged from 72 to 74.Source: §4.1 · p. 14
• DeepSeek-V3.2 narrows the open–closed-source gap on tool-use benchmarks but remains below frontier models, while improving over existing open models in unseen agent environments.Redundant self-verification can produce trajectories that exceed the 128K context limit, particularly on MCP-Mark GitHub and Playwright; context management can improve performance. The model achieved 𝜏2-bench category scores of 63.8 (Airline), 81.1 (Retail), and 96.2 (Telecom).Source: §4.1 · p. 14

4.2. Results of DeepSeek-V3.2-Speciale

• DeepSeek-V3.2-Speciale surpasses Gemini-3.0-Pro across multiple benchmarks and reaches gold-medal thresholds in IOI, ICPC WF, IMO, and CMO evaluations.It achieved IOI and ICPC WF results without targeted training; IMO and CMO performance used techniques from Shao et al. (2025). The reported CMO evaluation used the English version.Source: §4.2 · p. 14, Table 3 · p. 15, Table 4 · p. 15
• DeepSeek-V3.2-Speciale remains significantly less token-efficient than Gemini-3.0-Pro; stricter token constraints were used to balance performance against deployment cost and latency.The authors identify token efficiency as an area for future investigation.Source: §4.2 · pp. 14–15

4.3. Synthesis Agentic Tasks

• 12% accuracy on 50 sampled synthesized agentic tasks for DeepSeek-V3.2-Exp contrasts with at most 62% for frontier closed-source models, indicating the tasks challenged both.The results assess the difficulty of general synthesized tasks for both the model used in synthesis and frontier models.Source: §4.3 · p. 15, Table 5 · p. 16
• RL exclusively on synthetic agentic tasks improved DeepSeek-V3.2-SFT on Tau2Bench, MCP-Mark, and MCP-Universe, whereas RL limited to code and search did not.The synthetic-task RL used the SFT checkpoint in non-thinking mode, excluding long CoT and other RL data; comparisons included DeepSeek-V3.2-SFT and DeepSeek-V3.2-Exp, which was trained only in search and code environments.Source: §4.3 · pp. 15–16, Figure 5 · p. 16

4.4. Context Management of Search Agent

• On BrowseComp, Summary extends average steps to 364 with up to 60.2 performance improvement, while Discard-all scores 67.6, comparable to parallel scaling with fewer steps.Context management activates when token usage exceeds 80% of the context window; strategies summarize or discard prior tool-call history. Summary has relatively low efficiency, and the authors note that actual compute costs matter when comparing serial and parallel scaling.Source: §4.4 · pp. 16–17, Figure 6 · p. 17

Conclusion

• DeepSeek-V3.2 uses DSA to reduce computation while preserving long-context performance, achieves GPT-5-comparable reasoning, improves tool use, and its Speciale variant earns IMO and IOI gold medals.The paper attributes tool-use improvements to its large-scale agentic task synthesis pipeline and describes Speciale’s achievements as a milestone for open LLMs.Source: §5 · p. 17

Limitation, and Future Work

• DeepSeek-V3.2’s world knowledge lags leading proprietary models because it has fewer training FLOPs; future iterations will scale pre-training compute to address this gap.The comparison is with frontier closed-source models such as Gemini-3.0-Pro.Source: §5 · pp. 17–18
• DeepSeek-V3.2 typically needs longer generation trajectories than models like Gemini-3.0-Pro to match output quality; future work will optimize reasoning-chain intelligence density.Source: §5 · p. 18
• DeepSeek-V3.2 remains inferior to frontier models on complex tasks, motivating further refinement of its foundation model and post-training recipe.Source: §5 · p. 18

A. MHA and MQA Modes of MLA

• For DeepSeek-V3.1-Terminus, MHA is used for training and prefilling, while MQA is used for decoding.Source: Figure 7 · p. 20
• MLA has MHA and MQA modes, and Figure 7 illustrates their transformation.Source: A. MHA and MQA Modes of MLA · p. 20, Figure 7 · p. 20

B. Cold Start Template

• The reasoning data system prompt requires the model to output its reasoning process in <think></think> tags.This is the system-prompt requirement shown in Table 6.Source: Table 6 · p. 21
• The tool prompt replaces {TOOL-DESCRIPTIONS} and {TOOLCALL-FORMAT} with the specific tools and the designed tool-call format.These placeholders are replaced with the specific tools and the designed tool-call format.Source: Table 7 · p. 21
• The model executes tool calls within its thinking process.This is the behavior described in Table 8.Source: Table 8 · p. 21

• 39.3 is the non-thinking Terminal Bench 2.0 score with the Terminus framework.The table’s terminal benchmark scores are evaluated with the Claude Code framework.Source: Table 9 · p. 22
• Non-thinking mode is slightly worse than thinking mode but remains competitive.Source: C. Non-thinking DeepSeek-V3.2 Agentic… · p. 22

• All competitions used a 128k maximum generation length, no tools or internet access, and the contests’ time and attempt limits.Source: D. Evaluation Method of IOI, ICPC… · p. 22
• For IOI, 500 candidates per problem were filtered and the 50 longest-thinking valid solutions selected under the 50-submission limit and maximum-subtask scoring rule.Filtering removed candidates that failed sample tests, exceeded length constraints, or explicitly expressed inability or refusal to solve.Source: D. Evaluation Method of IOI, ICPC… · p. 22
• For ICPC, the IOI filtering methodology was applied to 32 generated candidates per problem.Source: D. Evaluation Method of IOI, ICPC… · p. 22
• For IMO and CMO, a generate-verify-refine loop revised solutions until perfect self-evaluation or the revision cap, following Shao et al. (2025).Source: D. Evaluation Method of IOI, ICPC… · p. 22