PaperlayerView exact revision on arXiv

Source-linked AI summary

One Sample to Rule Them All: Extreme Data Efficiency in Multidiscipline Reasoning with Reinforcement Learning

Yiyuan Li, Zhen Huang, Yanan Wu, Weixun Wang, Xuefeng Li, Yijia Luo, Wenbo Su, Bo Zheng, Pengfei Liu

arXiv:2601.03111v2cs.LGcs.CL

TL;DR

One-shot RL had been shown to improve math reasoning, but whether one sample could generalize across disciplines remained an open question. The paper introduces polymath learning, selecting or synthesizing one training sample for multidisciplinary reasoning. It reports cross-domain gains from a single sample and stronger gains from an engineered synthetic sample than from natural math samples.

  • Problem

    Whether a single math sample can improve reasoning across disciplines remained an open question, while multidisciplinary reasoning was relatively underexplored.

    Source: §1 · p. 1, §8 · p. 9

  • Method

    Polymath learning uses reinforcement learning with one carefully selected math sample or a synthesized multidisciplinary sample informed by salient math skills.

    Source: §1 · pp. 1–2

  • Results

    A single math sample yields gains across multiple domains, and an engineered synthetic sample achieves greater cross-domain gains than natural math samples.

    Source: §1 · p. 2, §8 · p. 9

  • Takeaways & Limitations

    The findings suggest deliberate sample selection and synthesis, or sample engineering, as a complement to simply increasing data volume.

    Source: §1 · p. 2

    Resource constraints limited experiments to a small set of samples, without larger-scale experiments in one-shot polymath learning.

    Source: A Limitations and Future Work · p. 16

Abstract

from arXiv · show

The reasoning ability of large language models (LLMs) can be unleashed with reinforcement learning (RL) (OpenAI, 2024; DeepSeek-AI et al., 2025a; Zeng et al., 2025). The success of existing RL attempts in LLMs usually rely on high-quality samples of large volumes. In this paper, we challenge conventional assumptions about data requirements in RL for LLMs by demonstrating the effectiveness of one-shot reinforcement learning. Specifically, we introduce polymath learning, a framework for designing one training sample that elicits multidisciplinary reasoning improvement. We present three key findings: (1) A single, strategically selected math reasoning sample can produce significant performance improvements across multiple domains, including physics, chemistry, and biology; (2) Analysis of salient mathematical skills provides insight into the characteristics associated with effective polymath samples; and (3) An engineered synthetic sample that integrates multidisciplinary elements and broader skill coverage achieves stronger performance than naturally occurring individual samples. Across various reasoning benchmarks, polymath learning achieves stronger performance than larger datasets, demonstrating that reasoning structure and skills in samples, rather than quantity, may be the key to unlock enhanced reasoning capabilities in language models. Our results suggest a shift, dubbed as sample engineering, toward precision engineering of samples that complements simply increasing data volume.

1 Introduction

The paper investigates whether one carefully chosen math sample can improve reasoning across disciplines, addressing the limited cross-domain evidence for one-shot RL. It also analyzes sample skills and proposes engineering a multidisciplinary synthetic sample.

1 Introduction

  • Prior one-sample RL evidence was preliminary and math-specific, leaving cross-domain generalization, optimal sample direction, and sample synthesis as open questions.Earlier work also found that 1,389 selected samples could outperform the full 8k-sample MATH dataset.Source: §1 · p. 1
  • A single math sample produces reasoning gains across physics, chemistry, biology, and other domains, exceeding comprehensive datasets with thousands of samples.The gains also reach domains described as semantically distant from mathematics.Source: §1 · pp. 1–2
  • Samples with broader coverage of foundational mathematical skills tend to yield stronger multidisciplinary gains, with algebra and precalculus prevalent across domains.The authors identify algebra and precalculus as broadly transferable reasoning components.Source: §1 · p. 2
  • A synthetic sample integrating multidisciplinary context and broad skill coverage yields greater cross-domain gains than natural math samples with narrower skill coverage.Further analysis attributes reasoning-power amplification to both internal multidisciplinary reasoning structure and skills, as well as computational complexity.Source: §1 · p. 2
  • The findings motivate sample engineering through deliberate selection and synthesis, alongside high-quality data reuse rather than simply increasing data volume.The authors note that scaling data volume may potentially induce generalization degradation.Source: §1 · p. 2

2 Related Work

Related work covers RL for reasoning, data efficiency, cross-domain transfer, and sample selection. The paper positions polymath learning as one-sample RL that extends reasoning transfer beyond math-focused settings.

2 Related Work

  • Rule-based verifiable rewards simplify critic-model implementation and can mitigate reward hacking in RL for long-form reasoning.RLVR rewards model outcomes that can be checked by verification functions.Source: §2 · p. 2
  • Prior data-efficient RL work selects responses, prompts, or training subsets, while one-sample and small-sample studies described here remain math-focused or require additional stages.Shrestha et al. (2025) use fewer than 100 samples but require pre-warmup distillation; Wang et al. (2025a) report one-sample math gains.Source: §2 · p. 2
  • Cross-domain reasoning studies include transfer from programming to math and RL transfer from math to other domains, but prior work leaves data efficiency or broader subject coverage unexamined.Li et al. (2025b) restrict their study to logical-intensive domains such as code and puzzles.Source: §2 · p. 2
  • Sample-selection approaches use gradients, quality scores, clustering, difficulty, role-play, and RL-dynamics alignment to estimate sample utility or diversity.Polymath learning selects samples using alignment with RL dynamics and salient math skills.Source: §2 · pp. 2–3

3 GRPO Basics

The section outlines GRPO’s group-based advantage estimation and explains how polymath learning applies it with a dataset containing one valid prompt-answer sample.

3 GRPO Basics

  • GRPO estimates each response’s advantage within a group of responses, replacing PPO’s critic model while retaining effectiveness.The group consists of responses generated for a prompt by the policy model.Source: §3 · p. 3
  • Each response’s reward is computed by applying a reward function to the response and the golden answer.The policy model’s token likelihood is conditioned on the prompt and preceding response tokens.Source: §3 · p. 3
  • Unlike approaches that assemble comprehensive datasets, polymath learning uses one valid sample, Dpolymath = (x1, ŷ1).Here x is the prompt and ŷ is the golden answer.Source: §3 · p. 3

4 Polymath Learning

Polymath learning uses one math sample selected for its relationship to RL training dynamics and salient skills, and synthesizes a multidisciplinary alternative through instruction-based generation and skill-based selection.

4 Polymath Learning

  • The paper situates one-sample polymath learning as an extension of earlier one-sample RL work focused on mathematical reasoning.The authors likewise conduct polymath learning from math reasoning problems.Source: §4 · p. 3
  • A preliminary model measures sample learnability from reward trajectories; candidates with the lowest LIMR scores (0.6) across math categories are selected to avoid risking math over-specialization.Higher LIMR scores indicate closer alignment between individual-sample behavior and dataset-wide reward behavior during RL training.Source: §4 · p. 3
  • Table 1 presents a natural algebra-category polymath sample as an example of the selected samples.Other samples are included in Appendix P.Source: Table 1 · p. 3, §4 · p. 3
  • The synthetic-sample pipeline generates multidisciplinary candidate problems and selects those with the most specialized salient math skills.Strong models generate candidates by integrating physics, chemistry, and biology; Qwen2.5-72Binstruct identifies salient skills for selection.Source: §4 · p. 4
  • The selected Synthetic Prime requires biology strand-sequence knowledge, chemistry bond energy, and physics photon-energy reasoning.It is chosen for its comprehensive skill spectrum.Source: §4 · p. 4, Table 2 · p. 4

5 Experimental Setup

The experiments use one selected polymath sample to train Qwen2.5-7b-base with GRPO and evaluate reasoning across mathematical and non-mathematical benchmarks. Table 3 compares natural and synthetic sample strategies across subject domains.

5 Experimental Setup

  • Qwen2.5-7b-base is trained with GRPO for 140 steps using a binary outcome reward and a batch of 128 prompts, with 16 responses generated per prompt.The selected sample is augmented into the batch; format reward and the KL term are excluded.Source: §5 · pp. 4–5
  • The benchmark suite covers MATH500, AIME 2024 and 2025, MinervaMath, GPQA-Diamond, SciBench, MMLU-Pro, and SuperGPQA.Responses use greedy single-attempt decoding except AIME, which averages 32 attempts at temperature 0.4.Source: §5 · p. 5
  • Table 3 compares sample strategies across subject domains; most synthetic specialists outperform their corresponding natural samples, and Synthetic Prime performs best.The caption says most natural polymath samples also outperform in-context learning and comprehensive learning with LIMR selection.Source: Table 3 · p. 5

6 Results

Polymath learning improves reasoning across subject domains, with synthetic samples—especially Synthetic Prime—showing strong cross-domain performance. Effective samples tend to cover broader salient mathematical skills, with algebra and precalculus recurring across domains.

6.1 Cross-Domain Generalization of Learning on Single Polymath Sample

  • Synthetic Prime achieves the strongest overall performance, while most synthetic specialists outperform their natural counterparts and show domain-specific advantages.The passage highlights Synthetic Prime's particular strength in physics and chemistry.Source: §6.1 · p. 6
  • Natural polymath samples generally surpass comprehensive learning on non-math subjects, while comprehensive MATH or LIMR training remains strong in math.LIMR consistently outperforms MATH in most subjects, but comprehensive learning lags behind the best polymath results across most non-math subjects.Source: §6.1 · p. 6
  • Polymath samples in prealgebra or algebra achieve performance on par with or above at least one comprehensive-learning model in over 50% of subjects.The best in-context polymath sample also outperforms the zero-shot pass-rate@64 baseline in most subjects.Source: §6.1 · p. 6
  • 14.5 points is the improvement over comprehensive learning on the full MATH set for the 50% of subjects farthest from MATH500, versus 7.7 points for the closest 50%.The passage interprets this pattern as stronger reasoning generalization in less math-intensive subjects.Source: §6.1 · p. 7

6.2 Characteristics of Polymath Sample in Salient Math Skills

  • Samples with broader salient foundational-skill coverage tend to yield stronger multidisciplinary gains, and synthetic samples with multidisciplinary context cover more skills than natural samples.Synthetic Prime has the highest concentration of salient skills; algebra and precalculus are especially prevalent across domains.Source: §6.2 · p. 7

7 Generalization of Self-Verification

Polymath learning shows more frequent self-verification than comprehensive learning, with different samples eliciting different verification behaviors. Programming's role varies across domains and does not necessarily yield reasoning gains without an external executor.

7 Generalization of Self-Verification

  • Self-verification is framed as a signal for reconsidering and refining initial solutions, and verification feedback can further enhance decision-making..Source: §7 · p. 7
  • Polymath learning generally elicits self-verification more frequently than comprehensive learning, with number theory prompting re-evaluation and intermediate algebra eliciting the most code use.The paper also reports distinct self-verification preferences across polymath samples.Source: §7 · p. 8, Figure 4 · p. 9
  • In math, programs are used mainly during final-answer generation, while in physics and chemistry they are used more for result validation.The authors note that program integration does not necessarily yield reasoning gains without access to an external executor.Source: §7 · p. 8

8 Conclusion

The paper argues that carefully designed single samples can improve reasoning across disciplines, while identifying study boundaries and implementation details. It also reports that polymath training with Synthetic Prime outperforms MATH-dataset comprehensive learning when using PPO.

8 Conclusion

  • Polymath learning with one sample can improve reasoning across disciplines; broader skill coverage and an engineered multidisciplinary sample are associated with stronger gains.The paper presents sample engineering as a complement to scaling data volume.Source: §8 · p. 9

A Limitations and Future Work

  • Experiments cover few samples, math-based samples, and open-ended formats, without scaled skill-based synthesis or a direct link between self-verification and reasoning gains.The study also does not extend to other domains where reliable rewards are accessible.Source: A Limitations and Future Work · p. 16

B LIMR Score Basics

  • LIMR compares each sample’s training reward in an epoch with the training set’s average reward for that epoch.Source: B LIMR Score Basics · p. 16

C Configurations

D Polymath Learning in other RL approach

E Disentangling the Synthetic Prime Sample

The Synthetic Prime variants separate mathematical reasoning from scientific context to examine their contributions. Results favor a LIMR score of 0.6 among the tested natural samples.

E Disentangling the Synthetic Prime Sample

  • The Synthetic Prime variants isolate its mathematical structure by removing scientific context, or retain scientific context while supplying intermediate results for simpler arithmetic.The math-only version uses symbolic replacements for chemistry and biology; the science-only version divides the problem into three milestone subproblems.Source: E Disentangling the Synthetic Prime… · pp. 16–17
  • Both variants outperform MATH-dataset training; science-only still improves multidisciplinary reasoning, while math-only matches Synthetic Prime on several domains.The authors suggest both non-trivial math reasoning and structured subject-specific formulation contribute to the sample’s effectiveness.Source: E Disentangling the Synthetic Prime… · p. 17, Table 10 · p. 18

F Sample Preference with LIMR Scores

G Results by Datasets

The synthetic sample performs strongest across the reported reasoning benchmarks, and the paper lists the subjects included in its evaluation.

G Results by Datasets

  • The synthetic sample performs strongest in the comparison of polymath learning and comprehensive learning across reasoning datasets.Table 12 covers open-ended and multiple-choice benchmarks.Source: G Results by Datasets · p. 19, Table 12 · p. 19

H Full Subject List

I Robustness of Experiments

Across three independent runs, Synthetic Prime polymath learning outperforms 8k-sample MATH training on most non-math benchmarks and overall performance. Verification patterns differ by subject domain.

I Robustness of Experiments

J Self-verification by Subject Domains

K Polymath Learning with Other 1-shot Sample

The supplementary analyses compare Synthetic Prime with other one-shot samples and larger training sets, trace training dynamics across model choices, and provide sample and reasoning examples.

K Polymath Learning with Other 1-shot Sample

  • Synthetic Prime consistently outperforms π1 and comprehensive learning on 8k MATH samples across Qwen2.5-7b-base and Qwen2.5-14b-base.Table 15 reports the comparison; the accompanying discussion also describes Synthetic Prime as outperforming both alternatives in those model sizes.Source: Table 15 · p. 23, K Polymath Learning with Other 1-shot… · p. 23

L Performance on MMLU-Pro and SuperGPQA Full Set

M Training Dynamics of Polymath Learning

  • Comprehensive learning improves MATH500 but overfits on multidisciplinary benchmarks, while polymath learning is more robust there despite lower MATH500 performance.Training on MATH exacerbates overfitting; Synthetic Prime and the natural prealgebra sample also outperform π1 on multidisciplinary benchmarks.Source: M Training Dynamics of Polymath Learning · p. 24, Figure 9 · p. 24

N Polymath Learning on Additional Models

  • Synthetic Prime works across most tested models; it does not surpass comprehensive learning on OctoThinker-8b-long-base but improves its GPQA-Diamond and SuperGPQA results.The comparisons include Qwen2.5-14b-base, Qwen3-8b-base, and OctoThinker-8b-long-base.Source: Table 17 · p. 25, N Polymath Learning on Additional Models · p. 25

P Other Polymath Learning Samples

Q Self-Verification Examples

O Reasoning Breakdown by Subject

  • Figure 10 orders subjects by embedding distance from MATH500 and contrasts base-model pass ratios with the best in-context and polymath results.The figure displays only the best polymath and in-context polymath results; “Synthetic” denotes the Synthetic Prime sample.Source: O Reasoning Breakdown by Subject · p. 25, Figure 10 · p. 26

R Example of Mathematical Skill in the Reasoning Problem

Loading 2601.03111v2…