Llama-Nemotron: Efficient Reasoning Models

• The named models are Llama-3.1-Nemotron-Nano-8B-v1, Llama-3.3-Nemotron-Super-49B-v1, Llama-3.1-Nemotron-Ultra-253B-v1, and Llama-3.1-Nemotron-Ultra-253B-CPT-v1.Source: NVIDIA · p. 1
• The section also names the Llama-Nemotron-Post-Training-Dataset.Source: NVIDIA · p. 1
• As of April 2025, Artificial Analysis identifies LN-Ultra as the most “intelligent” open model.This is the claim made in Figure 1's caption.Source: Figure 1 · p. 1
• LN-Ultra delivers leading performance among open models across a wide range of reasoning and non-reasoning benchmarks.This is the scope of Figure 2; no benchmark values are provided in the caption.Source: Figure 2 · p. 2

1. Introduction

• Reasoning models use long chains of thought, self-verification, reflection, and backtracking to achieve state-of-the-art results, but inference cost and verbosity make efficiency and user-controlled reasoning essential.The paper notes that detailed multi-step responses may be unnecessary or counterproductive for some queries.Source: §1 · p. 2
• LN-Ultra reportedly outperforms DeepSeek-R1, fits on one 8xH100 node with higher inference throughput, and was Artificial Analysis’s most intelligent open-source model as of April 2025.The Artificial Analysis ranking is stated as of April 2025.Source: §1 · pp. 2–3
• Derived from Llama 3.1 and 3.3, LN-Nano (8B), LN-Super (49B), and LN-Ultra (253B) target strong reasoning, efficient inference, enterprise use, and 128K-token contexts.The family is released under an open license for enterprise use.Source: §1 · pp. 2–3
• Each model uses a lightweight system prompt to switch dynamically between standard chat and reasoning, avoiding separate models or architectures while enabling detailed reasoning or cost-effective general use.The prompt is “detailed thinking on/off”.Source: §1 · p. 3
• Training uses NAS and FFN fusion, distillation and continued pretraining, SFT on instructions and teacher reasoning traces, large-scale math and STEM RL with FP8 generation, then brief alignment.For LN-Ultra, RL substantially boosts GPQA-D performance; the paper describes RL as enabling the student to surpass its teacher.Source: §1 · p. 3
• The released post-training dataset contains quality-, correctness-, and complexity-filtered synthetic responses for mathematical reasoning, coding, science, and instruction following.It supports supervised fine-tuning and reinforcement learning for all three model sizes.Source: §1 · p. 3

2. Creating Inference-Optimized Models

• Puzzle (Bercovich et al., 2024) distills alternative transformer blocks from Llama 3 Instruct models, then uses mixed-integer programming to select layer blocks under hardware, latency, memory, or throughput constraints.Its block library includes attention removal and variable-size FFNs; those operations offer accuracy-efficiency tradeoffs, and attention removal plus FFN compression were most effective for LN-Super and LN-Ultra in throughput and memory savings.Source: §2 · pp. 3–4
• FFN Fusion replaces consecutive FFN blocks created after attention removal with fewer, wider layers that run in parallel, reducing sequential depth without compromising expressivity.The technique is especially useful on multi-GPU setups, where inter-layer communication overhead is non-negligible.Source: §2 · p. 4

2.1. Deployment Constraints and Efficiency Targets

• LN-Super delivers 5× throughput over Llama 3.3-70B-Instruct at batch size 256 and TP1, and ≥2.17× over its optimal TP4 setup; LN-Ultra achieves 1.71× latency improvement after FFN Fusion.LN-Super targets approximately 300K cached tokens at FP8 on one H100; LN-Ultra supports up to 3M tokens at FP8 or 600K at BF16 on an eight-GPU H100 node.Source: §2.1 · pp. 4–5
• Across both GPQA-Diamond settings, LN-Ultra outperforms DeepSeek-R1 and Llama 3.1-405B in accuracy and throughput.Figure 4 compares accuracy and throughput at two input/output sequence settings, with 250 concurrent users.Source: Figure 4 · p. 5, §2.1 · p. 5

2.2. Post-NAS Training: Knowledge Distillation and Continued Pretraining

• After NAS, LN-Super receives 40B distillation tokens, while LN-Ultra receives 65B distillation tokens and 88B continued-pretraining tokens; this final step lets LN-Ultra match or surpass Llama 3.1-405B-Instruct on key benchmarks.The LN-Ultra comparison is after continued pretraining and before supervised fine-tuning and reinforcement learning.Source: §2.2 · p. 5, Table 1 · p. 6

3.1.1. Math

• Math problems come from AoPS forums; Qwen2.5-32B-Instruct extracts and classifies them, and proof, multiple-choice, binary, and invalid problems are removed.Middle School Math discussions are excluded as too easy and unhelpful in early experiments.Source: §3.1.1 · p. 6
• Math solutions are generated in reasoning and non-reasoning forms, benchmark-like questions are removed, and solutions are filtered against extracted or inferred answers.DeepSeek-R1 generates 16 reasoning solutions and Qwen2.5-Math-7B-Instruct generates 64 non-reasoning solutions per problem; when an answer cannot be extracted, the most common candidate answer serves as ground truth.Source: §3.1.1 · pp. 6–7

3.1.2. Code

• The code set contains 28,904 deduplicated competitive-programming questions; contamination checks found less than 0.3% overlap, and refinement produced approximately 488K Python samples.DeepSeek-R1 generates solutions, primarily in Python, with C++ for specific benchmark testing; post-processing checks reasoning traces, extracts code, and validates syntax.Source: §3.1.2 · p. 7
• Scaling code data from 25k to 736k samples continuously improved coding-benchmark performance; focusing first on harder CodeContests problems yielded the largest gains.The scaling curve did not plateau.Source: §3.1.2 · p. 7

3.1.3. Science

• Science data combines StackOverflow question-answer pairs with topic- and difficulty-conditioned synthetic MCQs and variants, decontaminates against major benchmarks, and generates reasoning traces with majority-vote answers when labels are absent.Synthetic questions are verified for format compliance.Source: §3.1.3 · pp. 7–8

3.1.4. General

• General-domain data pairs synthetic prompts across open and closed QA, extraction, and brainstorming with permissively licensed real-world prompts; multiple responses are rejection-sampled using a reward model.The process follows NVIDIA (2024c) and uses the Llama-3.1-Nemotron-70B reward model.Source: §3.1.4 · p. 8

3.2. Reasoning off

• Training pairs each prompt with reasoning and non-reasoning responses, tags them with corresponding system instructions, filters responses against ground-truth answers or reward models, and augments data with permissive function-calling and safety datasets.Non-reasoning responses are generated with Llama-3.1-Nemotron-70B-Instruct for general-domain prompts and Llama-3.3-70B-Instruct for others.Source: §3.2 · p. 8

3.2.1. General-Domain Open-ended Inference-Time Scaling

• A feedback-edit pipeline turns 20k first-turn ShareGPT and WildChat prompts into selected high-quality responses through feedback, targeted editing, and response selection.Llama-3.1-Nemotron-70B-Instruct generates initial responses; dedicated Feedback, Edit, and Select models refine them.Source: §3.2.1 · p. 8

4.1. General Methodology

• SFT uses token-level cross-entropy on mixed reasoning/non-reasoning instruction data; higher learning rates aid long traces, while multi-epoch training helps smaller models and warmup with cosine decay stabilizes training.The authors note that sequence-length-dependent token-loss averaging makes long reasoning traces harder to learn from, and that warmup was crucial for LN-Ultra.Source: §4.1 · p. 9

4.2. Model-Specific Training

• LN-Nano uses three SFT stages: reasoning-only code, math, and science data to prevent repetitive completions; mixed reasoning/non-reasoning data for control; then chat, instruction-following, and tool-calling.Training uses sequence packing with effective sequence length 32k and global batch size 256; the first stage uses a 1e−4 learning rate for four epochs.Source: §4.2 · p. 10
• LN-Super trains one epoch on the full SFT set (5e−6 learning rate, 16k sequence length, batch 256); compute limits curtailed promising 3–4-epoch, 5e−5 runs, and rejection fine-tuning gave no gains.Source: §4.2 · p. 10
• LN-Ultra packs 24k-token sequences at batch 256; warmup to 1e−5 and cosine decay to 1e−6 mitigated high-rate instability, but post-epoch-one explosions required optimizer reinitialization before convergence.Source: §4.2 · p. 10

5. RL for Reasoning

• Reasoning RL enabled LN-Ultra to surpass DeepSeek-R1 and set a new state of the art on GPQA among open models.The paper says supervised fine-tuning let LN-Ultra approach, but not exceed, DeepSeek-R1; resource constraints and preliminary results led the authors to apply reasoning RL exclusively to LN-Ultra.Source: §5 · p. 10

5.1. Training Procedure

• LN-Ultra used GRPO with 72-prompt rollouts, 16 responses per prompt, temperature=1, top_p=1, batch size 576, and two gradient updates per rollout.Accuracy rewards used Llama-3.3-70B-Instruct to judge answers against ground truth, while format rewards enforced thinking tags according to the reasoning toggle. Training filtered out prompts with LN-Super pass rates of 0.75 or higher and used progressive batching from easier to harder examples.Source: §5.1 · pp. 10–11

5.2.1. Overview

• RL training used NeMo-Aligner, with vLLM handling generation and Megatron-LM handling training on the same GPUs.The stages shared nodes to improve GPU utilization, making memory management important for the 253B-parameter model.Source: §5.2.1 · pp. 11–12, §5.2.2 · p. 12

5.2.2. Memory Profiling and Optimizations

• The selected TP=8, CP=2, PP=18, DP=2 configuration with sequence parallelism achieved >90% GPU utilization without GPU or CPU out-of-memory errors.Heterogeneous model weights created a 13B-element tensor occupying 26B of GPU memory during weight preparation; FP8 generation loosened vLLM’s memory budget enough to re-enable cudagraphs.Source: §5.2.2 · p. 13

5.2.3. FP8 Inference Generation

• 32 tokens/s/GPU/prompt peak FP8 generation throughput delivered 1.8x speedup over BF16.The reported speedup comprises 1.4x from FP8 generation and an additional 0.4x from reduced memory use, which enabled vLLM’s cudagraph feature.Source: §5.2.3 · p. 13

6.1. Instruction Following

• Less than 120 RLOO steps with a verifier reward and batch size 128 improve conventional instruction-following and reasoning benchmarks.LN-Super and LN-Ultra are trained on synthetic prompts containing one to ten detailed instructions, using a verification setup similar to Zhou et al. (2023) and RLOO (Ahmadian et al., 2024).Source: §6.1 · p. 13

6.2. RLHF

• 88.3: LN-Super (49B) achieves this Arena Hard score, beating Claude 3.5 Sonnet, GPT-4o-2024-05-13, Llama-3.1-405b-instruct, and Mistral-large-2407.RLHF targets general helpfulness and chat capabilities while maintaining proficiency in other areas.Source: §6.2 · p. 14
• 69.1 to 88.1: two online RPO iterations increase Arena Hard, while improving all other adopted benchmarks except IFEval.The authors speculate that RLHF helps the model better use its existing knowledge and skills, since neither the dataset nor reward model targets math, coding, science, or function calling.Source: §6.2 · p. 14
• Online RPO maximizes reward-model predictions over HelpSteer2 prompts for LN-Super, using two 500-step iterations.The procedure uses Llama-3.1-Nemotron-70B-Reward, with RPO (NVIDIA, 2024c; Sun et al., 2025), learning rate 4e-7, KL penalty 1e-5, reward scale 3.0, and batch size 64.Source: §6.2 · p. 14
• LN-Ultra uses GRPO, sampling 8 responses per prompt and training for 30 steps.Training uses learning rate 3e-7, batch size 288, and KL penalty 1e-3.Source: §6.2 · p. 14
• LN-Nano undergoes two offline RPO rounds with on-policy data: the first targets reasoning control, and the second targets instruction following.The first round mixes reasoning and non-reasoning data with appropriate system prompts; each round trains up to 400 steps with learning rate 7e-7, KL penalty 3e-2, and batch size 512.Source: §6.2 · p. 14

7.1. Benchmarks

• The evaluation covers AIME24, AIME25, GPQA-Diamond, LiveCodeBench, MATH500, IFEval, BFCL V2 Live, and Arena-Hard, using a 32k context length.LN-Nano uses AIME25-I only, while LN-Super and LN-Ultra use the full 30-question AIME25 set; reasoning-heavy scores can vary across runs because of small datasets and generation randomness.Source: §7.1 · pp. 14–15

7.2. LN-Nano

• LN-Nano performs strongly across reasoning benchmarks, including AIME25-I and LiveCodeBench, with chemistry upsampling improving GPQA-D and RPO mainly targeting IFEval.The reported results attribute its performance to the SFT pipeline and curated reasoning data, with balanced math, coding, and STEM coverage.Source: §7.2 · p. 15, Table 3 · p. 15

7.3. LN-Super

• LN-Super performs competitively across both modes, matching Llama-3.3-70B in reasoning-off mode and outperforming DeepSeek-R1-Distilled-Llama-70B in reasoning-on mode.Improving IFEval can compromise Arena-Hard conversationality, so model merging selected a Pareto-frontier checkpoint; LiveCodeBench weakness is attributed to training on an earlier dataset version.Source: §7.3 · pp. 15–16, Table 4 · p. 16

7.4. LN-Ultra

• LN-Ultra matches or outperforms existing open-weight models across reasoning and non-reasoning benchmarks and achieves state-of-the-art GPQA performance among open models.It is optimized to run on a single 8×H100 node, unlike DeepSeek-R1, which requires 8×H200.Source: §7.4 · p. 16, Table 5 · p. 16
• LN-Ultra-SFT approaches DeepSeek-R1 on several reasoning benchmarks, while RL is critical to surpassing it, particularly on GPQA.Although higher-scoring SFT checkpoints were available, RL began from an earlier checkpoint to improve final RL outcomes.Source: §7.4 · p. 16

8. Evaluations on Judging Capability

• The evaluation tests out-of-distribution LLM-as-a-Judge capability on JudgeBench, which asks models to differentiate high-quality from low-quality responses.Source: §8 · p. 17
• LN-Ultra is the best open-source JudgeBench model, significantly surpassing DeepSeek-R1 and trailing only o3-mini(high); LN-Super also outperforms o1-mini.Table 6 reports the JudgeBench comparison.Source: Table 6 · p. 17, §8 · p. 17

9. Conclusions

• The models are released under a permissive license, with weights, training data, and code open-sourced.Source: §9 · p. 17
• The Llama-Nemotron models perform competitively with state-of-the-art reasoning models while requiring little memory and enabling efficient inference.Source: §9 · p. 17
• With a strong reasoning teacher, supervised fine-tuning on high-quality synthetic data is effective at adding reasoning capabilities to smaller models.Source: §9 · p. 17
• To push reasoning beyond what a teacher model alone can provide, large-scale curriculum-driven reinforcement learning from verifiable rewards is necessary.Source: §9 · p. 17
• Producing an all-around model that performs well across varied benchmarks requires several stages in the post-training pipeline.Source: §9 · p. 17

Post-training

• The first Post-training passage lists contributor names beginning with Jiaqi Zeng, Soumye Singhal, and Alexander Bukharin.It continues with additional names, including Yian Zhang, Tugrul Konuk, Gerald Shen, and Ameya Sunil Mahabaleshwarkar.Source: Post-training · p. 17
• A second Post-training passage lists further contributors, beginning with Ameya Sunil Mahabaleshwarkar, Bilal Kartal, and Yoshi Suhara.The list also includes Olivier Delalleau, Zijia Chen, Zhilin Wang, David Mosallanezhad, Adi Renduchintala, Haifeng Qian, Dima Rekesh, and Fei Jia.Source: Post-training · p. 18

Data

• The Data passage contains a contributor-name list beginning with Somshubra Majumdar, Vahid Noroozi, and Wasi Uddin Ahmad.It continues through Roger Waleffe.Source: Data · p. 18

Infrastructure

• The Infrastructure passage contains a contributor-name list beginning with Jimmy Zhang, Bor-Yiing Su, and Guyue Huang.The list continues through Robert McQueen.Source: Infrastructure · p. 18

Inference

• The first Inference passage lists contributors beginning with Izzy Putterman, George Lam, and Arun Venkatesan.The name list continues through Shahar.Source: Inference · p. 18
• A second Inference passage continues the contributor-name list, beginning with Mor, Omer Ullman Argov, and Scot Junkin.The list continues through Joyjit Daw.Source: Inference · p. 18

Evaluations and Safety

• Evaluations and Safety lists Soumye Singhal*, Denys Fridman, Erick Galinkin, Michael Evans, Shaona Ghosh, Katherine Luna, Leon Derczynski, and Nikki Pope.Source: Evaluations and Safety · p. 18
• Evaluations and Safety also lists Eileen Long, Seth Schneider, Guillermo Siman, Tomasz Grzegorzek, Pablo Ribalta, Monika Katariya, and Chris Alexiuk.Source: Evaluations and Safety · p. 18

Program management

• Program management lists Joey Conway*, Ehud Karpas*, Trisha Saar*, Ann Guan, Krzysztof Pawelec, and Shyamala Prayaga.Source: Program management · p. 18
• The listed Program management contributors are Joey Conway*, Ehud Karpas*, and Trisha Saar*.Source: Program management · p. 18

Leadership

• Leadership lists Tugrul Konuk*, Oleksii Kuchaiev*, Boris Ginsburg*, Oluwatobi Olabiyi*, and Kari Briski*.Source: Leadership · p. 18
• Leadership also lists Jonathan Cohen*, Bryan Catanzaro*, Jonah Alben, Yonatan Geifman, and Eric Chung.Source: Leadership · p. 18