PaperlayerView exact revision on arXiv

Source-linked AI summary

Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models

Boxin Wang, Chankyu Lee, Nayeon Lee, Sheng-Chieh Lin, Wenliang Dai, Yang Chen, Yangyi Chen, Zhuolin Yang, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping

arXiv:2512.13607v2cs.CLcs.AIcs.LG

TL;DR

General-purpose RL must handle domains with different response lengths and verification costs, complicating infrastructure and training design. Nemotron-Cascade uses sequential, domain-wise RL, and its unified 8B model closes the thinking-performance gap with a dedicated 8B-Thinking model.

  • Problem

    RL across domains varies in response length and reward-computation latency, complicating infrastructure, curriculum design, and hyperparameter tuning.

    Source: §1 · p. 4, §8.1 · p. 38

  • Method

    Cascade RL trains models sequentially across domains rather than jointly blending diverse prompt distributions.

    Source: §1 · p. 5

  • Results

    Cascade RL sets state-of-the-art results across multiple domains, and the unified 8B model closes the thinking-performance gap with a dedicated 8B-Thinking model.

    Source: §1 · pp. 5–6

  • Takeaways & Limitations

    RLHF substantially improves response quality and thereby enhances reasoning performance.

    Source: §1 · p. 5

    RewardBench is an imperfect proxy for selecting the best aligned policy, and RLHF introduces additional variance.

    Source: §4.2.2 · p. 16

Abstract

from arXiv · show

Building general-purpose reasoning models with reinforcement learning (RL) entails substantial cross-domain heterogeneity, including large variation in inference-time response lengths and verification latency. Such variability complicates the RL infrastructure, slows training, and makes training curriculum (e.g., response length extension) and hyperparameter selection challenging. In this work, we propose cascaded domain-wise reinforcement learning (Cascade RL) to develop Nemotron-Cascade, capable of operating in both instruct and deep thinking modes, without any performance gap relative to a thinking-only counterpart. Departing from conventional approaches that blend heterogeneous prompts from different domains, Cascade RL orchestrates sequential, domain-wise RL, reducing engineering complexity and delivering state-of-the-art performance across a wide range of benchmarks. Notably, RLHF for alignment, when used as a pre-step, boosts the model's reasoning ability far beyond mere preference optimization, and subsequent domain-wise RLVR stages rarely degrade the benchmark performance attained in earlier domains and may even improve it (see an illustration in Figure 1). Our 14B model, after RL, outperforms its SFT teacher, DeepSeek-R1-0528, on LiveCodeBench v5/v6/Pro and achieves silver-medal performance in the 2025 International Olympiad in Informatics (IOI). We transparently share our training and data recipes.

1. Introduction

Nemotron-Cascade scales sequential, domain-wise reinforcement learning into an open post-training recipe for unified models that operate in thinking and instruct modes. The introduction presents this approach as a way to manage cross-domain training heterogeneity while preserving strong reasoning performance.

Motivation

  • Domain-specific response lengths and reward-verification latency complicate RL infrastructure, slow training, and make curriculum design and hyperparameter selection harder.The introduction contrasts fast rule-based math verification with slower code execution verification and reward-model scores for alignment and creative writing.Source: §1 · p. 4

Prior work

  • Earlier cascaded math-then-code RL work found that math training improved math and code benchmarks, while later code training boosted code results without degrading math.The paper uses this prior result to motivate scaling the cascaded paradigm across more domains.Source: §1 · p. 4

Cascade RL

  • Cascade RL trains sequentially across human-feedback alignment, instruction following, math, competitive programming, and software engineering rather than blending prompts across domains for joint RL.The authors present the framework as an open recipe built from Qwen3-8B-Base and Qwen3-14B-Base, with training and data recipes shared.Source: §1 · pp. 5–6
  • RLHF improves response quality and reasoning, while later domain-specific RL stages rarely degrade earlier-domain benchmarks and may improve them; curricula and hyperparameters can be tailored by domain.The introduction gives reduced verbosity as an example of RLHF’s improvement to response quality.Source: §1 · pp. 5–6

Unified reasoning modes

  • The 8B unified model closes the thinking-performance gap with a dedicated 8B-Thinking model despite also training on non-thinking data.The stated techniques are SFT curation of paired thinking and instruct responses for the same prompts, and RLHF batches with equal prompt shares from both modes.Source: §1 · p. 6

Results

  • With a 64K-token inference budget, the 14B dedicated Thinking model outperforms Gemini-2.5-Pro-06-05, o4-mini (medium), Qwen3-235B-A22B (thinking mode), and DeepSeek-R1-0528 on LiveCodeBench v5/v6, and earns silver-medal performance at IOI 2025.The paper identifies DeepSeek-R1-0528 as the model’s SFT teacher.Source: §1 · p. 6

2. Main Results

The Cascade RL models achieve best-in-class performance across almost all benchmarks, with substantial improvements over their SFT starting points. The results also show strong competitive-coding and software-engineering performance for general-purpose models.

Main Results

  • Evaluation covers general-knowledge, instruction-following, math, coding, software-engineering, and tool-use benchmarks, using a 64K-token generation limit and temperature 0.6 and top-p 0.95 for reasoning tasks.Baseline results use official reports when available.Source: §2 · p. 6
  • The paper evaluates models after each Cascade RL stage, with results summarized for SFT, RLHF, instruction-following RL, math RL, code RL, and SWE RL.These stage-wise results are in Tables 2 and 4–8, respectively.Source: §2 · pp. 6–7
  • Nemotron-Cascade-8B and Nemotron-Cascade-14B-Thinking achieve best-in-class performance across almost all benchmarks, and the complete Cascade RL pipeline substantially improves the SFT models.Scores are reported as pass@1, averaged over k generations per prompt.Source: §2 · p. 6, Table 1 · p. 7
  • Nemotron-Cascade-8B scores 74.3 on LCB v5 and 71.1 on LCB v6, compared with DeepSeek-R1-0528 (671B) at 74.8 and 73.3, respectively.The paper describes the 8B model's performance as highly comparable despite DeepSeek-R1-0528 serving as its SFT teacher.Source: §2 · p. 7
  • Nemotron-Cascade-14B-Thinking surpasses DeepSeek-R1-0528 across all splits of LCB and LCB Pro.Additional results, including IOI 2025, are provided in Section 5.Source: §2 · p. 7
  • On SWE-bench Verified, Nemotron-Cascade-8B and 14B attain 37.2% and 43.1% pass@1 accuracy, respectively, versus 42.2% for specialized DeepSWE-32B.DeepSWE-32B is built on Qwen3-32B and specialized for software-engineering tasks.Source: §2 · p. 7

3. Supervised Fine-Tuning

The section presents a two-stage SFT pipeline spanning general and reasoning domains, with explicit controls for thinking and non-thinking modes. After SFT, the 8B unified model matches the dedicated 8B Thinking model on reasoning benchmarks and surpasses it on IFEval.

3.1.1. Multi-Stage SFT

  • SFT proceeds in two one-epoch stages: 16K-token general, math, science, and code data, followed by 32K-token reasoning, tool-use, and software-engineering data.General-domain prompts have parallel thinking and non-thinking responses; other domains use thinking-mode responses.Source: §3.1.1 · p. 8

3.1.2. Chat Template

  • Per-user-turn /think and /no_think flags enable global consistency or mode switching within a conversation; tool calls use system-listed tools and dedicated tags.The method uses ChatML and relies on explicit flags for thinking-mode control.Source: §3.1.2 · pp. 8–9, Figure 3 · p. 8

3.2.1. General-Domain Data

  • The general-domain corpus contains 2.8M samples comprising 3.2B tokens, while knowledge-intensive questions add 1.2M samples comprising 1.5B tokens.To improve consistency and quality, the authors generate parallel thinking and non-thinking responses, retain some original high-quality answers, and filter generations using ground truth or an auxiliary model.Source: §3.2.1 · p. 9

3.2.2. Math Reasoning Data; 3.2.3. Code Reasoning Data

  • Math and code reasoning datasets use thinking-mode responses, with Stage 2 extending the context limit to 32K tokens and adding longer reasoning or challenging coding prompts.The Stage-1 sources use 16K-token limits; math Stage 2 excludes relatively easy questions, while code data draws on separate Stage-1 and Stage-2 prompt sets.Source: §3.2.2 · p. 10, §3.2.3 · p. 10

3.2.4. Science Reasoning Data

  • Science SFT retains prompts requiring knowledge and substantial reasoning, adds rarer question types, and uses thinking-mode responses in both stages.The Stage-2 science data are upsampled by 2× before blending.Source: §3.2.4 · pp. 10–11

3.2.5. Tool Calling Data

  • Stage-2 tool-calling SFT covers single-turn, multi-turn, and multi-step interactions, using 310K conversations comprising 1.41M user-assistant turns.Responses are generated by Qwen3-235B-A22B, and all tool-calling data use thinking mode.Source: §3.2.5 · p. 11

3.3. Software Engineering Task; 3.3.1. Agentless Framework; 3.3.2. Data Curation

  • The software-engineering SFT uses a simplified Agentless workflow, targeted diff-style repair patches, and filtered task data with repository-level deduplication.The workflow focuses localization on identifying relevant files; SFT data are selected using task-specific quality checks, and the resulting SWE datasets are upsampled by 3× for Stage 2.Source: §3.3.1 · p. 11, §3.3.2 · pp. 12–13

3.4. Results after SFT

  • The 8B unified model performs on par with the dedicated 8B Thinking model across reasoning benchmarks and surpasses it on IFEval.Both use the same SFT thinking data, while the unified model also incorporates non-thinking data; the 14B Thinking model provides stronger results than the 8B models.Source: §3.4 · p. 13

4. Cascade RL

This section describes Cascade RL’s staged training approach, beginning with RLHF and proceeding through domain-specific RLVR stages in math, code, and SWE. It covers the associated training recipes, outcomes, and reported effects on other benchmarks.

4.1.1. Why Cascade RL for LLMs Is Resistant to Catastrophic Forgetting

  • Cascade RL sequences RLHF, instruction-following RL, Math RL, Code RL, and SWE RL, relying on policy-generated experience, overlapping rewards, limited prompt overlap, and general-to-specialized ordering to mitigate forgetting.SFT and RL prompts are strictly disjoint; the RLHF stage also excludes math and competitive-programming prompts to reduce cross-domain interference.Source: §4.1 · p. 13, §4 · p. 13, §4.1.1 · p. 14

4.1.2. RL Training Configuration

  • Across Cascade RL, GRPO uses on-policy rollouts and a single update per group, making the importance-sampling ratio exactly 1; removing KL divergence simplifies the objective and supports stable training.The setup uses group-normalized rewards and token-level loss; RLVR rewards can be verified against ground truth, while RLHF uses reward-model scores.Source: §4.1.2 · pp. 14–15

4.2. Reward Modeling

  • The 82K-pair preference dataset combines open-source and in-house data; the Qwen2.5-72B-Instruct reward model uses Bradley–Terry training, while RewardBench is an imperfect proxy for downstream policy alignment.A single epoch yielded the best results, and the model with the highest RewardBench score did not necessarily produce the best-aligned policy.Source: §4.2.1 · p. 15, §4.2.2 · pp. 15–16

4.2.2. Training Recipe

  • Reward-model performance scaled positively with backbone size; WorldPM helped early, but vanilla Qwen2.5-72B-Instruct later caught up and slightly surpassed it, while Qwen3 backbones underperformed Qwen2.5 counterparts.Larger models were more robust to stylistic artifacts, and Qwen3 models in non-thinking mode still did not surpass Qwen2.5-Instruct.Source: §4.2.2 · p. 16, Table 3 · p. 17

4.3.3. Results after RLHF

  • RLHF improves nearly every reported benchmark except IFEval, while penalizing verbose and repetitive generations; IFEval declines amid prompt overlap and reward-model preference conflicts.The authors report a 2% AIME25 drop in early experiments when math prompts were not excluded from RLHF, and curate RLHF data without math or competitive-programming prompts.Source: §4.3.1 · p. 17, §4.3.3 · p. 18, Table 4 · p. 19

4.4.2. Training Recipe

  • IF-RL uses progressively harder IFEval- then IF-Bench-taxonomy stages with dynamic filtering; for unified models, applying it only in non-thinking mode achieved IFEval 85.3 in thinking mode.A rule-based verifier alone could degrade human alignment, motivating a combined verifier and preference-reward approach for dedicated thinking models.Source: §4.4.2 · pp. 19–20

4.4.3. Results after IF-RL

  • IF-RL significantly improves IFEval and IFBench, with controlled small ArenaHard degradation; reasoning-token lengths fall, and most reasoning-benchmark losses recover after later RL stages.The unified reasoning model achieves a stronger balance across ArenaHard and IFEval than the dedicated thinking model.Source: §4.4.3 · p. 20, Table 5 · p. 21

4.5.2. Training Recipe

  • Math RL initialized from RLHF-trained models improves initial reasoning and token efficiency and reduces training steps; its recipe reached around 90% AIME24 across five 8B checkpoints within 500 steps.The math data is filtered for benchmark contamination and unsuitable or noisy problems, with answer verification and removal of problems solvable by the 7B model at high success rates.Source: §4.5.1 · pp. 20–21, §4.5.2 · p. 21
  • Math RL extends reasoning budgets through 24K, 32K, and 40K stages, using compression and stabilization before longer reasoning.The final 40K stage improves hard AIME accuracy from 30 to 40% while other problem performance remains high; the initial 24K stage brings diverse models to a more consistent reasoning-length range.Source: §4.5.2 · p. 22
  • Epoch-based math filtering removes problems solved at 100% or 0% accuracy, then re-samples hard problems at 10% and easy problems at 1%.This keeps approximately 90% of samples contributing learning signals and stabilizes accuracy, especially later in training.Source: §4.5.2 · pp. 22–23

4.6.1. Data Curation

  • Code RL follows Math RL and uses a curated 9.8K-sample coding set filtered for test coverage, duplicates, contamination, and difficulty.Training uses binary all-tests-pass rewards and asynchronous verification, reducing verification time from 1172.4 to 416.2 seconds per batch in the stated 8-node setup.Source: §4.6.1 · pp. 23–24, §4.6.2 · p. 24

4.5.3. Results after Math RL

  • Math RL improves AIME 2024/2025 and coding benchmarks while having minimal effect on knowledge-reasoning and alignment benchmarks.Most observed differences on knowledge-reasoning and alignment benchmarks are attributed to evaluation variance and checkpoint selection.Source: §4.5.3 · p. 23, Table 6 · p. 24

4.6.3. Results after Code RL

  • 75.3 on LCB v5 and 71.5 on LCB v6 for unified 8B match DeepSeek-R1-0528 (74.8 and 73.3); 14B-Thinking scores 78.0 and 74.8.Code RL has minimal impact on benchmarks from other domains, aside from checkpoint and evaluation variance.Source: Table 7 · p. 25, §4.6.3 · p. 25

4.7.1. Data Curation

  • SWE repair data retains challenging prompts and trains on both ground-truth-only and mixed-localization contexts, with noisy files added under a length limit.Prompts are selected using response similarity thresholds, include up to five files, and are discarded if shorter than 8K tokens.Source: §4.7.1 · p. 26

4.7.2. Training Recipe

  • SWE RL uses an execution-free patch verifier and expands input context from 16K to 24K while keeping output length at 16K.Its reward combines lexical similarity with LLM-assessed semantic similarity; a two-stage curriculum is intended to avoid degradation from immediate long-context training, particularly for 8B models.Source: §4.7.2 · pp. 26–27

4.7.3. Results after SWE RL

  • 43.1 pass@1 resolve rate on SWE-bench Verified lets 14B-Thinking outperform DeepSWE-32B (42.2) and SWE-agent-LM-32B (40.2).After the full Cascade RL process, unified 8B scores 38.5 versus 37.2 for 8B-Thinking on SWE-bench Verified; unified 8B performs comparably on reasoning tasks and better on instruction-following tasks.Source: §4.7.3 · pp. 27–28, Table 8 · p. 28

5. Deep Dive on Competitive Coding

Nemotron-Cascade performs strongly across competitive coding benchmarks, and its 14B-Thinking model earns a silver medal at IOI 2025 using feedback-driven test-time scaling. The section also examines training temperature and how successive Cascade RL stages affect coding reasoning.

5. Deep Dive on Competitive Coding

  • Nemotron-Cascade-14B-Thinking outperforms DeepSeek-R1-0528, its SFT teacher, and two larger Qwen models across competitive coding benchmarks, while the 8B model matches OpenReasoning-Nemotron-32B.The 8B model also outperforms nearly all recently released reasoning LLMs of comparable size.Source: §5 · p. 28, Table 9 · p. 29

5.1. Test-Time Scaling in Practice: IOI 2025

  • The IOI pipeline uses a 128K-token thinking budget, up to 50 rounds, and 20 candidates per subtask per round, filtering submissions and selecting with Tail-10.Each round incorporates official-judge feedback: prompts retain up to five recent submissions for classic problems or three top-scoring submissions for partial-score problems; solved subtask code also informs other subtasks.Source: §5.1 · p. 29
  • 343.37 points earned Nemotron-Cascade-14B-Thinking a silver medal at IOI 2025; on Problem 2 Triples, it scored 90.37 points.The Problem 2 score exceeded OpenAI’s internal IOI-gold model (75.29 points) and DeepSeek-V3.2-Speciale (82 points); the full contest used at most 1,000 generations and 50 submissions per problem.Source: Figure 5 · p. 30, §5.1 · p. 30

5.2. The Role of Training Temperature in Code RL

  • Higher temperatures improved code-reasoning performance and sample efficiency, while lower temperatures stabilized entropy; high temperatures could also cause entropy explosion.The ablation compared temperatures 0.6, 0.8, and 1.0 on the 8B unified model.Source: §5.2 · p. 30, Figure 6 · p. 30

5.3. How Cascade RL Improves Code Reasoning

  • RLHF improved accuracy while reducing verbosity; IF-RL cut reasoning-token use 20% with a 0.5% accuracy drop, and Code RL lifted medium- and hard-problem performance.Easy-problem accuracy saturated above 99% after the initial stages; Math RL increased token use and improved medium-problem accuracy.Source: Figure 8 · p. 31, §5.3 · p. 31
  • Topic-wise, RLHF improved all coding categories, Math RL mainly helped Math, Graph, and Geometry, and Code RL delivered the largest gains across nearly all topics.Math RL yielded limited improvement on String and Data Structure problems.Source: §5.3 · p. 31

6. Deep Dive on RLHF

The section finds that reward-model size and training-mode choices shape RLHF outcomes, while stabilization techniques are best used when training shows signs of instability.

6.1. RLHF Training Strategies for Unified Models

  • Half-Half RLHF training produced the strongest overall performance, improving ArenaHard, math, and code results despite evaluation in thinking mode.Half-Half splits each batch evenly between thinking and non-thinking prompts, outperforming training in either mode alone.Source: Figure 9 · p. 32, §6.1 · p. 32

6.2. Impact of Reward Model Size on RLHF Performance

  • Around 3% higher AIME25 accuracy with the 72B versus 7B reward model; code differences stayed within 1%, while 72B gave the highest ArenaHard score under style control.Smaller reward models were more prone to reward hacking through longer outputs, and higher RewardBench scores did not always predict better ArenaHard performance.Source: Figure 10 · p. 32, §6.2 · p. 33

6.3. Bag of Tricks for Stablizing RLHF Training

  • With the 7B reward model, stabilization techniques extended stable RL training from 350 to 950 steps; with 72B, training was stable without them.The toolbox includes KL penalty, sequence-level loss aggregation, and reward shaping that normalizes rewards and applies tanh; the authors recommend using these techniques only when instability appears.Source: §6.3 · pp. 33–34, Table 10 · p. 34, Table 11 · p. 34

7. Deep Dive on SWE

The SWE deep dive improves code localization, execution-free reward design, long-context repair training, and test-time patch selection. Its results show benefits from complementary localization signals, semantic rewards, suitable prompt lengths, and validation-based test-time scaling.

7.1. Generation–Retrieval Approach for Code Localization

  • Localization combines generation from issue descriptions and repository structure with NV-Embed-Code retrieval over source contents, then fuses rankings using reciprocal rank fusion with k=0.The generation approach aggregates multiple rollouts and ranks files by frequency of appearance.Source: §7.1 · p. 34
  • Retrieval outperformed generation for localization, while multi-rollout aggregation improved generation rankings and reciprocal rank fusion added slight gains, particularly below top-5 cutoffs.The authors attribute retrieval’s advantage to its use of full repository source code, compared with generation’s reliance on repository structure.Source: §7.1 · pp. 34–35, Figure 11 · p. 35

7.2. Execution-Free Reward Model for SWE RL

  • Code-repair RL uses an execution-free reward comparing generated patches with human-written golden patches through lexical or semantic similarity.Lexical similarity uses Unidiff; semantic similarity is scored by Kimi-Dev-72B. The ablation evaluates both with ground-truth localized files and with the top four files from generation–retrieval localization.Source: §7.2 · p. 35
  • RL generally improved code repair, semantic similarity rewards outperformed lexical similarity, and threshold shaping helped lexical but not semantic rewards.Setting rewards to zero below 0.5 improved lexical reward effectiveness, suggesting it filters noisy supervision; the default SWE RL setup used semantic similarity without shaping (condition 3).Source: §7.2 · pp. 35–36

7.3. Improving Long-Context Analysis

  • Training on longer prompts improved repair from 16K to 32K, but 40K was less effective; final maximum prompt lengths were 24K for 8B and 32K for 14B.Resolve rates dropped significantly when prompt length exceeded 24K alongside a 16K response; the authors hypothesize limits from SFT’s 32K sequence length and noisy trajectories or limited pretrained long-context capability at 40K.Source: §7.3 · p. 36, Table 13 · p. 36

7.4. Test-Time Scaling and Patch Validation

  • The best@k test-time pipeline ranks candidate patches by regression-test and curated reproduction-test pass rates, breaking ties by majority vote and then shorter solutions.For SWE-bench Verified, it generates k patches and 40 reproduction tests per instance; the reported results show best@k consistently outperforming majority@k.Source: §7.4 · pp. 36–37, Figure 12 · p. 37
  • 43.6% was Nemotron-Cascade-8B’s best@32 resolve rate, up from 39.2% at k=2; Nemotron-Cascade-14B-Thinking’s best@k plateaued around 53.8%.At k=32, the 8B model’s pass@k score was 57.7%; the 14B model began at a majority@k resolve rate of 50.7%.Source: Figure 12 · p. 37, §7.4 · p. 37

8. Related Work · Appendix

The related-work review positions Cascade RL among prior approaches to RL for language models, SFT and distillation, and unified reasoning models. It contrasts domain-wise RL with joint training across heterogeneous tasks and outlines how the paper’s approach addresses that challenge.

8.1. Reinforcement Learning for LLMs

  • RLHF aligns language models after SFT, while reward-model RL for math reasoning has faced challenges in modeling mathematical rewards.Prior work describes reward models as surrogates for human judges; RLVR instead uses objective, deterministic criteria such as symbolic verification.Source: §8.1 · p. 38
  • Open RLVR recipes have advanced reasoning, but their models focus mainly on math and code rather than general-purpose frontier capabilities.Examples include AceReason-Nemotron (Chen et al., 2025; Liu et al., 2025), DeepScaleR (Luo et al., 2025), DeepCoder (Luo et al., 2025), DAPO (Yu et al., 2025), and Skywork-OR1.Source: §8.1 · p. 38
  • Cascade RL addresses the infrastructure, curriculum, and hyperparameter challenges of joint training across heterogeneous tasks with domain-wise RL for general-purpose models.The authors also release training recipes and datasets and investigate the interplay between RLHF and RLVR across math, coding, science, instruction following, software engineering, and general-domain tasks.Source: §8.1 · p. 38

8.2. Supervised Fine-Tuning and Distillation

  • SFT prepares pretrained models for conversation and instruction following, while distillation trains smaller models from teacher outputs or synthetic responses.Off-policy distillation can reuse generated data after its costly creation, and on-policy distillation has been explored to reduce the gap to on-policy RL (Yang et al., 2025).Source: §8.2 · pp. 38–39
  • Prior work found that well-designed RL can narrow initial SFT performance gaps when exploration and exploitation are appropriately balanced (Liu et al., 2025).This work framed the finding as synergy between SFT and RL.Source: §8.2 · p. 39

8.3. Unified Reasoning Models

  • Recent thinking models emphasize deep reasoning through long chain-of-thought generation, including analysis, strategy exploration, verification, and correction.Examples include OpenAI’s o1 (OpenAI, 2024), o3 and o4-mini (OpenAI, 2025), DeepSeek-R1 (Guo et al., 2025), and Qwen3-Thinking.Source: §8.3 · p. 39
  • Unified models offer different mode controls: Llama-Nemotron uses a system prompt, Qwen3, GLM-4.5, and DeepSeek-V3.1 allow turn-level switching, while GPT-5 routes automatically.The paper characterizes automatic routing as circumventing rather than resolving the challenge of unifying instruct and thinking modes.Source: §8.3 · p. 39

B. Benchmarks and Evaluation Setups

The section defines knowledge, alignment, math, and code benchmarks, specifying evaluation modes, metrics, sampling procedures, and inference settings for Cascade and baseline models.

Knowledge reasoning

  • Knowledge reasoning uses MMLU, MMLU-Pro, and GPQA-Diamond, with evaluation in thinking mode.MMLU has 14,079 questions across 57 subjects and MMLU-Pro has over 12,000 questions; both use exact match from one generation per question. GPQA-Diamond has 198 graduate-level science questions and reports pass@1 averaged over 8 generations; Cascade uses temperature 0.6, top-p 0.95, and a 64K-token budget with YaRN scaling factor 2.Source: B. Benchmarks and Evaluation Setups · pp. 39–40

Alignment tasks

  • Alignment evaluation covers verifiable instruction following and human-preference benchmarks, with model mode varying by task.IFEval uses prompt strict and avg@8 pass@1; the unified model is evaluated in non-thinking mode and the dedicated thinking model in thinking mode, unlike prior work’s instruct strict metric (NVIDIA, 2025). IFBench uses thinking-mode avg@8 pass@1, while ArenaHard 1.0 uses thinking mode and GPT4-Turbo-2024-0409 as judge; Cascade uses temperature 0.6, top-p 0.95, and a 32K-token response limit, and baselines use official results when available or recommended/same settings.Source: B. Benchmarks and Evaluation Setups · p. 40

Math reasoning

  • Math reasoning is evaluated on AIME 2024 and AIME 2025, each containing 30 problems.Cascade uses a 64K-token thinking budget, temperature 0.6, top-p 0.95, and YaRN scaling factor 2; baselines use recommended settings with at least a 64K-token budget.Source: B. Benchmarks and Evaluation Setups · p. 40

Code generation

  • Code evaluation spans LiveCodeBench, LiveCodeBench Pro, and SWE-bench Verified, using thinking-mode pass@1 accuracy.LiveCodeBench v5/v6 contain 279/454 problems, and LiveCodeBench Pro 2025Q1/Q2 contain 166/167; these use avg@8, while SWE-bench Verified’s 500 samples use avg@4. Cascade uses 64K-token budgets for LiveCodeBench and Pro and 32K for SWE-bench Verified; baselines use recommended settings with at least 64K tokens for LiveCodeBench and Pro.Source: B. Benchmarks and Evaluation Setups · pp. 40–41

C. Prompt Templates

The templates define how to generate and judge deliberately similar but incorrect off-topic prompts, and provide a coding prompt with reference materials for IOI 2025 test-time scaling.

Step 1: Generate offtopic prompts

C.3. Prompt Templates for Test-Time Scaling on IOI 2025

  • The IOI 2025 prompt asks for a Python solution in a specified code format for the supplied problem statement.Source: §C.3 · p. 45
  • The IOI prompt provides accepted solutions with different input constraints as reference material for insights only.It includes accepted code and notes that the constraints differ.Source: §C.3 · p. 45
  • The prompt also supplies submission history with incorrect code and official verdicts and scores, which it identifies as reliable.It instructs the solver to use the history to improve the solution when helpful.Source: §C.3 · pp. 45–46

D. Training Hyperparameters · D.1. Multi-Stage SFT

The section identifies the multi-stage SFT hyperparameters for the 8B and 14B models and states that unified and thinking models share them.

D.1. Multi-Stage SFT

D.2. RLHF

The section presents RLHF training hyperparameters for the 8B and 14B models. Unified and thinking models share the same hyperparameters, while unified models split prompts equally between non-thinking and thinking modes.

D.2. RLHF

  • Unified and thinking models share RLHF hyperparameters; unified models allocate prompts equally to non-thinking and thinking modes.The section presents the RLHF hyperparameters for the 8B and 14B models in Table 15.Source: Table 15 · p. 46, §D.2 · p. 46

D.3. IF-RL

This section reports the IF-RL training hyperparameters for the 8B and 14B models, with unified models trained in non-thinking mode.

D.3. IF-RL

  • Table 16 covers IF-RL training hyperparameters for the 8B and 14B models and specifies that unified models use non-thinking mode.Source: Table 16 · p. 47, §D.3 · p. 47

D.4. Math RL · D.5. Code RL · D.6. SWE RL

The appendix documents training hyperparameters for Math, Code, and SWE RL, noting when unified models share settings and are trained in thinking mode.

D.4. Math RL

D.5. Code RL

D.4. Math RL

  • Table 18 covers Math RL training hyperparameters for the 14B-Thinking model.Source: Table 18 · p. 48

D.5. Code RL

  • Table 19 notes that the 8B unified Code RL model is trained in thinking mode.Source: Table 19 · p. 48

D.6. SWE RL

  • SWE RL hyperparameters for 8B unified, 8B-Thinking, and 14B-Thinking models are listed in Table 20.Source: §D.6 · p. 49, Table 20 · p. 49
  • Table 20 states that 8B unified and thinking models share SWE RL hyperparameters, with unified models trained in thinking mode.Source: Table 20 · p. 49

E. ELO Rating Analysis

The section estimates Codeforces Elo ratings for Nemotron-Cascade-8B and Nemotron-Cascade-14B-Thinking across 51 contests, and finds substantial contest-to-contest variability in the 14B-Thinking model’s performance.

E. ELO Rating Analysis

  • Ratings were estimated from 51 recent contests using up to eight independent submissions per problem, with expected submission penalties and standard contest scoring rules.Responses used temperature 0.6, top-p 0.95, and a 128K-token maximum; the reported Elo is averaged across the rounds.Source: E. ELO Rating Analysis · pp. 49–50
  • Performance details for the 8B and 14B-Thinking models are reported in Tables 21 and 22, respectively.The tables cover 51 Codeforces rounds from 2501–2507, using eight attempts per problem and estimated human rank and Elo.Source: E. ELO Rating Analysis · p. 50, Table 21 · p. 51, Table 22 · p. 52
  • The 14B-Thinking model’s estimated rating ranged from above 2600 on Round 1015 to below 1000 after solving no problems on Round 1024 Div.1, and its problem-solving behavior was inconsistent.It often handled standard techniques, heavy implementation, or straightforward intuition, but struggled with hypothesis-driven exploration, ad hoc ideas, constructive problems, and interactive problems.Source: E. ELO Rating Analysis · p. 50

Related papers

Loading 2512.13607v2…