PaperlayerView exact revision on arXiv

Source-linked AI summary

Tiny Model, Big Logic: Diversity-Driven Optimization Elicits Large-Model Reasoning Ability in VibeThinker-1.5B

Sen Xu, Yi Zhou, Wei Wang, Jixin Min, Zhibin Yin, Yingwei Dai, Shixi Liu, Lianyu Pang, Yirong Chen, Junlin Zhang

arXiv:2511.06221v1cs.AIcs.CL

TL;DR

The paper asks whether compact models can match large-model reasoning without relying on extreme parameter scaling. It uses diversity-oriented supervised fine-tuning and uncertainty-guided reinforcement learning; VibeThinker-1.5B beats DeepSeek R1-0120 on three math benchmarks and reports a strong coding result.

  • Problem

    The paper questions the prevailing assumption that strong reasoning requires extreme parameter scaling and notes that the optimal interface between SFT and RL remains under-explored.

    Source: §1 · p. 2, §3.1 · p. 4

  • Method

    The approach builds a diverse solution spectrum through two-stage diversity-exploring distillation, then uses MaxEnt-guided policy optimization to prioritize uncertain problems and amplify effective reasoning paths.

    Source: §1 · p. 2, §3.2 · p. 5, §3.4 · p. 6

  • Results

    VibeThinker-1.5B surpasses DeepSeek R1-0120 on AIME24 (80.3 vs. 79.8), AIME25 (74.4 vs. 70.0), and HMMT25 (50.4 vs. 41.7), and scores 51.1 vs. GPT-4.1's 44.7 on LiveCodeBench v6.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The report presents a 1.5B model as evidence that algorithmic design can support reasoning capabilities comparable to models tens to hundreds of times larger.

    Source: §1 · p. 2

    A substantial 20–40 point gap remains between VibeThinker-1.5B and leading models on GPQA, a broad general-knowledge benchmark.

    Source: §4.2 · p. 11

Abstract

from arXiv · show

Challenging the prevailing consensus that small models inherently lack robust reasoning, this report introduces VibeThinker-1.5B, a 1.5B-parameter dense model developed via our Spectrum-to-Signal Principle (SSP). This challenges the prevailing approach of scaling model parameters to enhance capabilities, as seen in models like DeepSeek R1 (671B) and Kimi k2 (>1T). The SSP framework first employs a Two-Stage Diversity-Exploring Distillation (SFT) to generate a broad spectrum of solutions, followed by MaxEnt-Guided Policy Optimization (RL) to amplify the correct signal. With a total training cost of only $7,800, VibeThinker-1.5B demonstrates superior reasoning capabilities compared to closed-source models like Magistral Medium and Claude Opus 4, and performs on par with open-source models like GPT OSS-20B Medium. Remarkably, it surpasses the 400x larger DeepSeek R1 on three math benchmarks: AIME24 (80.3 vs. 79.8), AIME25 (74.4 vs. 70.0), and HMMT25 (50.4 vs. 41.7). This is a substantial improvement over its base model (6.7, 4.3, and 0.6, respectively). On LiveCodeBench V6, it scores 51.1, outperforming Magistral Medium's 50.3 and its base model's 0.0. These findings demonstrate that small models can achieve reasoning capabilities comparable to large models, drastically reducing training and inference costs and thereby democratizing advanced AI research.

1 Introduction

The introduction questions whether reasoning ability requires extreme parameter scaling and presents diversity-driven post-training as an alternative. VibeThinker-1.5B reports strong math and coding results against much larger models.

1 Introduction

  • The introduction situates the work amid large reasoning models, where reinforcement learning and test-time scaling are presented as key strategies for improving reasoning accuracy.It names OpenAI o1[20], DeepSeek-R1[12], Qwen3[35], and GLM 4.5[37] in this context.Source: §1 · p. 2
  • The report challenges the view that logical reasoning requires scaling to models such as DeepSeek R1 (671B), Kimi-K2, and Qwen3-Max (>1T).It frames compact-model reasoning as a way to reduce training and inference costs, energy use, and environmental impact.Source: §1 · p. 2
  • VibeThinker-1.5B's Spectrum-to-Signal Principle separates SFT's diverse-solution generation from RL's selection and amplification of effective reasoning paths.The introduction says MaxEnt-Guided Policy Optimization prioritizes problems where the model is highly uncertain.Source: §1 · p. 2
  • With 1.5B parameters, VibeThinker-1.5B beats DeepSeek R1-0120 on AIME24 (80.3 vs. 79.8), AIME25 (74.4 vs. 70.0), and HMMT25 (50.4 vs. 41.7).Figure 2 also reports 74.4 for VibeThinker-1.5B versus 72.1 for GPT-OSS-20B-Medium and 74.0 for Seed-Thinking v1.5 on AIME25.Source: §1 · p. 2, Figure 2 · p. 3
  • On LiveCodeBench v6, VibeThinker-1.5B scores 51.1 vs. GPT-4.1's 44.7, while its 1.5B parameters are 100× to 600× fewer than those of cited giants.The report describes its reasoning capabilities in mathematics, code, and scientific tasks as comparable to models tens to hundreds of times larger.Source: §1 · p. 2
  • The authors connect competitive small models with lower research costs and broader participation beyond organizations with extensive compute resources.They describe broader participation as conditional on small models demonstrating competitive performance across multiple domains.Source: §1 · pp. 2–3

2 Preliminaries

The preliminaries define supervised fine-tuning and group-relative policy optimization, then connect output diversity and Pass@K to exploration and verifiable success.

2 Preliminaries

  • SFT adapts a pretrained language model to labeled downstream tasks by minimizing cross-entropy, equivalent to maximizing the likelihood of reference responses.The text says this enhances task-specific alignment while preserving pretrained knowledge.Source: §2 · p. 3
  • GRPO replaces critic-based advantage estimation with group-relative reward statistics and optimizes a clipped surrogate objective over sampled responses.A KL-divergence penalty relative to a reference policy may be added for stability.Source: §2 · pp. 3–4
  • Output diversity is response variability for an input; exploring multiple reasoning paths can help problem-solving robustness and creativity in math and code.The text contrasts diverse responses with repetitive or suboptimal outputs that limit handling of complex tasks.Source: §2 · p. 4
  • Pass@K measures the probability that at least one of k independently generated solutions passes verification, with correctness represented by a binary reward.The text reports a positive correlation between solution diversity and potential performance gains after RL training.Source: §2 · p. 4

3 Methodology

The section proposes a diversity-first SFT-to-RL pipeline, then details how specialist checkpoints and uncertainty-focused weighting implement it. It also reports benchmark, decontamination, and cost evidence for the resulting compact model.

3.1 The Spectrum-to-Signal Principle for SFT-RL Synergy

  • SSP rejects Pass@1-only SFT checkpoint selection: SFT expands the Pass@K solution spectrum, and RL amplifies correct answers from that pool.Source: §3.1 · p. 4

3.2 Training Pipeline

  • SFT builds a diverse solution spectrum through probing and fusion; MGPO then trains on uncertain math problems across 16K and 32K contexts, followed by code generation.Source: §3.2 · p. 5

3.3 Diversity-Exploring Distillation

  • Domain-aware probing selects high-Pass@K checkpoints for each subdomain, then expert fusion combines them into one SFT model.Source: §3.2 · p. 5, §3.3 · pp. 5–6
  • The fused SFT model improves both Pass@K diversity and Pass@1 accuracy, indicating that broader solution coverage need not reduce single-answer accuracy.Source: §3.3 · p. 6

3.4 MaxEnt-Guided Policy Optimization (MGPO)

  • MGPO gives greatest weight to questions near 0.5 accuracy and scales GRPO advantages; λ controls weighting sharpness, with λ = 0 recovering standard GRPO.Source: §3.4 · pp. 6–7

3.5 Training Data & Decontamination

  • Training uses mostly public open-source data plus some proprietary synthetic data, with normalization and 10-gram semantic matching to reduce train-evaluation overlap.Source: §3.5 · p. 7
  • VibeThinker scores 74.4 on AIME25 versus DeepSeek R1’s 70.0, 50.4 on HMMT25 versus 41.7, and 51.1 on LiveCodeBench v6 versus Magistral medium’s 50.3.Source: §3.5 · p. 8

3.6 Training Cost

  • Post-training costs under $8K (1/30 to 1/60 of DeepSeek R1 and MiniMax-M1) accompany 20 to 70 times lower inference costs and edge-device deployment.Source: Table 1 · p. 8, §3.6 · p. 8

4 Evaluation and Analysis

The evaluation compares VibeThinker-1.5B with small reasoning models, much larger reasoning models, and top-tier non-reasoning models across mathematics, coding, and specialized knowledge. Results report strong mathematical and coding comparisons, alongside a persistent general-knowledge gap.

4.1 Experimental Setup

  • Evaluation covers MATH-500, HMMT 2025, AIME 2024/2025, LiveCodeBench V5/V6, and GPQA-Diamond; scores use sampled Pass@1 estimates, with benchmark-specific sample counts.For LiveCodeBench V6, this evaluation uses the 131-problem February–May 2025 definition, while some comparison models may use a 454-problem definition.Source: §4.1 · pp. 8–9

4.2 Evaluation Results

  • Against its base model, Qwen2.5-Math-1.5B, AIME25 rises from 4.3 to 74.4, HMMT25 from 0.6 to 50.4, and GPQA from 16.4 to 46.7.LiveCodeBench V5 reaches 55.9 from a baseline of 0.Source: §4.2 · p. 9
  • VibeThinker-1.5B surpasses DeepSeek R1-0120 across all three mathematical datasets and exceeds Magistral Medium and Claude Opus 4 on AIME24 and AIME25.Its scores are also comparable to O3-mini-Medium and Gemini 2.5 Flash, and closely aligned with MiniMax-M1.Source: Table 3 · p. 10, §4.2 · p. 10
  • Among sub-3B models, VibeThinker-1.5B leads on AIME25 against Qwen3-1.7B (74.4 vs. 36.8) and on LiveCodeBench V6 (51.1 vs. 26.9).The comparisons also report advantages over SmolLM 3B on AIME25 (74.4 vs. 36.7), HMMT25 (50.4 vs. 26.0), and LiveCodeBench V5 (55.1 vs. 27.6).Source: §4.2 · pp. 9–10
  • VibeThinker-1.5B surpasses all compared non-reasoning models on challenging mathematics benchmarks and outperforms most in code generation.The table covers core benchmarks for top-tier non-reasoning models.Source: Table 4 · p. 11, §4.2 · p. 11
  • A 20-40-point GPQA gap with leading models persists; coding lags mathematics more, which the authors attribute primarily to the math-focused base model's limited code exposure.The authors posit that strengthening the base model's code capabilities could elevate VibeThinker's performance.Source: §4.2 · p. 11

5 Conclusion

VibeThinker-1.5B challenges the prevailing scaling paradigm with strong reasoning performance at a fraction of the cost. The report attributes this to post-training innovation that enhances output diversity during supervised fine-tuning and reinforcement learning.

5 Conclusion

  • Under $8,000, VibeThinker-1.5B surpasses DeepSeek R1 on AIME25 and outperforms other leading large models on key benchmarks.The report presents these results as evidence that compact models can achieve state-of-the-art reasoning performance at a fraction of the cost.Source: §5 · p. 11
  • VibeThinker-1.5B's post-training enhances output diversity during supervised fine-tuning and reinforcement learning rather than relying on extreme parameter scaling.Source: §5 · p. 11
  • The results indicate that small models can possess formidable reasoning capabilities, prompting a re-evaluation of Scaling Law assumptions.Source: §5 · p. 11
Loading 2511.06221v1…