PaperlayerView exact revision on arXiv

Source-linked AI summary

P1: Mastering Physics Olympiads with Reinforcement Learning

Jiacheng Chen, Qianjia Cheng, Fangchen Yu, Haiyuan Wan, Yuchen Zhang, Shenghe Zheng, Junchi Yao, Qingyang Zhang, Haonan He, Yun Luo, Yufeng Zhao, Futing Wang, Li Sheng, Chengxing Xie, Yuxin Zuo, Yizhuo Li, Wenxauan Zeng, Yulun Wu, Rui Huang, Dongzhan Zhou, Kai Chen, Yu Qiao, Lei Bai, Yu Cheng, Ning Ding, Bowen Zhou, Peng Ye, Ganqu Cui

arXiv:2511.13612v1cs.LGcs.AIcs.CL

TL;DR

LLMs must move beyond rubric-fitting toward reasoning that withstands physical constraints, and the paper addresses this challenge with open-source P1 models trained through reinforcement learning and augmented with agentic inference. P1-235B-A22B reaches Gold-medal performance at IPhO 2025, while P1 with PhysicsMinions achieves No.1 overall on the IPhO and HiPhO leaderboards.

  • Problem

    Science-grade reasoning requires outputs consistent with physical reality, making physics a demanding test of whether LLMs can reason beyond rubric-fitting.

    Source: §1 · p. 3

  • Method

    P1 is an open-source physics reasoning model family trained through RL and paired at inference with agentic control for adaptive reasoning.

    Source: §1 · pp. 3–4

  • Results

    P1-235B-A22B earned 12 golds and 1 silver across the HiPhO suite and Gold-medal performance on IPhO 2025; with PhysicsMinions, P1 ranked No.1 overall on IPhO 2025 and HiPhO.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors frame Olympiad-level physics mastery as a milestone toward LLMs that may eventually assist or pioneer real-world physics research.

    Source: §6 · p. 17

    Model-based verification can reward incorrect answers, and the authors argue that reliable judgment precision is needed before expanding its recall.

    Source: §5.2 · p. 16

Abstract

from arXiv · show

Recent progress in large language models (LLMs) has moved the frontier from puzzle-solving to science-grade reasoning-the kind needed to tackle problems whose answers must stand against nature, not merely fit a rubric. Physics is the sharpest test of this shift, which binds symbols to reality in a fundamental way, serving as the cornerstone of most modern technologies. In this work, we manage to advance physics research by developing large language models with exceptional physics reasoning capabilities, especially excel at solving Olympiad-level physics problems. We introduce P1, a family of open-source physics reasoning models trained entirely through reinforcement learning (RL). Among them, P1-235B-A22B is the first open-source model with Gold-medal performance at the latest International Physics Olympiad (IPhO 2025), and wins 12 gold medals out of 13 international/regional physics competitions in 2024/2025. P1-30B-A3B also surpasses almost all other open-source models on IPhO 2025, getting a silver medal. Further equipped with an agentic framework PhysicsMinions, P1-235B-A22B+PhysicsMinions achieves overall No.1 on IPhO 2025, and obtains the highest average score over the 13 physics competitions. Besides physics, P1 models also present great performance on other reasoning tasks like math and coding, showing the great generalibility of P1 series.

1. Introduction

The paper treats physics Olympiads as a rigorous test of science-grade reasoning and introduces open-source P1 models trained with reinforcement learning and agentic inference. P1-235B-A22B reaches gold-medal performance at IPhO 2025, and P1 with PhysicsMinions leads the IPhO and HiPhO 2025 leaderboards.

1. Introduction

  • Olympiad problems are presented as a high-fidelity standardized test of scientific reasoning, and mastery is framed as a necessary step toward future physics research.The paper describes competitive problem-solving as a milestone, not the end goal.Source: §1 · p. 3
  • P1 is an open-source model family trained through RL post-training, with adaptive multi-stage training and agentic control at inference.PhysicsMinions supports multi-turn reflection so models can critique and refine solutions.Source: §1 · pp. 3–4
  • P1-235B-A22B earned 12 golds and 1 silver across the 13-exam HiPhO suite and became the first open model with Gold-medal performance on IPhO 2025.The P1-30B-A3B variant attained Silver-medal performance on IPhO 2025.Source: §1 · p. 3
  • With PhysicsMinions, P1 attained the No.1 overall score on both the IPhO 2025 and HiPhO leaderboards.PhysicsMinions adds iterative correction and self-verification at inference.Source: §1 · p. 3
  • Physics post-training results on reasoning benchmarks suggest transferable reasoning skills rather than domain overfitting.The 30B variant is also reported to outperform its base model across seven math, coding, and general reasoning benchmarks.Source: §1 · pp. 3–4

2. Physics Dataset

The paper introduces a curated, high-difficulty physics corpus for reasoning under physical constraints, then details its source selection, annotation, and quality-control pipeline.

2.1. Overview

  • The dataset contains 5,065 problems: 4,126 from physics Olympiads and 939 from competition textbooks, spanning 5 fields and 25 subfields.The authors prioritize depth and rigor over broader coverage.Source: §2.1 · p. 4
  • Each item follows a Question–Solution–Answer schema with metadata, expert-authored solutions, rule-verifiable answers, and physics-field and source labels.Long problems may be subdivided to fit model context limits while retaining task integrity.Source: §2.1 · p. 5

2.2. Dataset Construction

  • The corpus combines ten major Olympiads with ten authoritative competition textbooks, selected for rigorous modeling, symbolic precision, and multi-step inference.Olympiads provide a regional-to-international difficulty spectrum, while textbooks contribute organized examples and exercises with expert-authored solutions.Source: §2.2 · pp. 5–6
  • Annotation uses OCR, source-specific question and solution parsing, answer extraction and decomposition, and English normalization; quality control yields 5,065 rule-verifiable text-only problems.The pipeline removes problems requiring diagram drawing or involving unverifiable answers, and reduces the collection from 6,516 items.Source: §2.2 · p. 6

3. Approach

P1 formulates physics problem-solving as token-level reinforcement learning and combines sequence-level optimization, adaptive training adjustments, and inference-time agentic review. Its training setup addresses learnability and engine mismatch, while validation performance improves across training stages.

3.1. RL Formulation

  • Physics problems are modeled as token-generation trajectories whose expected reward is optimized with policy gradients and sequence-level GSPO advantages.GSPO uses length-normalized sequence likelihood ratios.Source: §3.1 · pp. 6–7

3.2. Instantiation

  • Binary answer-correctness rewards are averaged across required sub-answers, with symbolic rule checks complemented by an LLM verifier during validation.The model is prompted to place each sub-answer in a separate boxed environment for extraction.Source: §3.2 · pp. 7–8

3.3.1. Adaptive Learnability Adjustment

  • Pass-rate filtering removes overly easy (>0.7) and unsolved (=0) tasks; group size and generation windows expand as model capability grows.Larger groups increase the chance of sampling informative trajectories on difficult problems, while longer windows accommodate deeper reasoning.Source: §3.3.1 · pp. 9–10

3.3.2. Training Stabilization Mechanism

  • Different rollout and training engines can bias gradients and destabilize training; truncated importance sampling corrects distribution shift while capping weight variance.The truncation operator prevents excessively large importance weights.Source: §3.3.2 · p. 10

3.4. Training Dynamics

  • Training stages expand group size and generation windows while applying train-inference correction; response lengths and IPhO validation results rise during training.The pipeline starts from Qwen3-30B-A3B-Thinking and Qwen3-235B-A22B-Thinking and periodically resumes from checkpoints with updated configurations.Source: §3.4 · pp. 10–12

3.5. Agentic Augmentation

  • PhysicsMinions combines a Logic solver and introspector with dual verifiers, retrying failed solutions (CV=2); P1’s text-only models disable the Visual Studio.The Physics-Verifier checks physical consistency, while the General-Verifier reviews reasoning and calculations.Source: §3.5 · p. 12

4. Experiment

The experiments evaluate P1 on recent physics Olympiads using HiPhO’s exam-score protocol and compare its models with other systems. P1 performs strongly both alone and with PhysicsMinions, including gold-level results on IPhO 2025 and CPhO 2025.

4.1. Experimental Setup

  • HiPhO covers 13 recent Olympiads from 2024–2025 across seven competition types, and the evaluation averages eight inference runs per problem.Scores combine answer-level and step-level grading against official marking schemes, with partial credit for valid intermediate reasoning; three competitions were excluded for lacking complete official contestant scores.Source: §4.1 · pp. 12–14

4.2. Evaluation on Physics Olympiads

  • P1-235B-A22B earns 12 gold and 1 silver across HiPhO, while P1-30B-A3B earns 8 gold, 4 silver, and 1 bronze.On IPhO 2025, P1-235B-A22B scores 21.2 / 30, ranks Top 3 globally, and is the first open-source model to reach gold-medal performance; P1-30B-A3B ranks third among open-source models.Source: Table 3 · p. 13, §4.2 · p. 14
  • 35.9 to 38.4: P1 with PhysicsMinions reaches overall Top-1 across models and leads on four physics Olympiads.It scores 23.2 vs. 22.7 on IPhO 2025, 28.0 vs. 27.9 on APhO 2025, 23.5 vs. 23.4 on EuPhO 2024, and 77.5 vs. 72.1 on PanMechanics 2025.Source: §4.2 · p. 14
  • 227 / 320: P1-235B-A22B exceeds the top-1 human medalist’s 199 on the CPhO 2025 theoretical exam.Human experts assessed the exam using its official marking scheme.Source: §4.2 · p. 14, Table 4 · p. 14

5. Discussion

P1’s domain-focused post-training improves general reasoning performance on many evaluated tasks, while direct use of a model-based verifier during training can lengthen responses and degrade validation performance. The discussion attributes verifier risks to false positives and argues that precision must be secured before expanding recall.

5.1. Generalizability of P1

  • P1-30B-A3B outperforms its base model across all reported metrics, while P1-235B-A22B is superior in most reported categories.The larger model’s reported wins include AIME24, AIME25, GPQA, HLE, IMO-AnswerBench, AMOBench, and overall average; Figure 6 summarizes comparisons across math and code datasets.Source: §5.1 · p. 15, Figure 6 · p. 15
  • The authors hypothesize that refined reasoning trajectories and structural similarities between physics problems and other tasks contribute to P1’s transfer to math, STEM, and coding.They specifically note advanced mathematics benchmarks such as IMO-style problems.Source: §5.1 · p. 15

5.2. Rule-based vs Model-based Verifier for Training

  • A model-based verifier during training increases response length while validation performance deteriorates relative to rule-only verification.The authors attribute the risk partly to exploitable verifier preferences and to false positives that can destabilize reinforcement learning.Source: §5.2 · p. 16, Figure 7 · p. 16
  • The rule-based verifier offers high precision but limited recall, whereas the model-based verifier expands recognition at the cost of incorrect judgments.False positives are described as more detrimental than false negatives during reinforcement learning because high-reward errors can dominate the learning signal.Source: §5.2 · p. 16
  • The authors argue that model-based verification should expand recall only after achieving a stable, reliable judgment boundary with sufficiently high precision.They identify comprehensive, fully correct rule-based verification as difficult and describe their design as combining rule-based and model-based reasoning for broader coverage.Source: §5.2 · pp. 15–16

6. Conclusion

The paper presents P1 as an open-source model family for Olympiad-level physics reasoning, trained with reinforcement learning and paired with agentic inference. It reports strong physics results and performance on math and coding, framing Olympiad mastery as a step toward AI assistance in physics research.

6. Conclusion

  • P1 is presented as the first family of open-source models capable of mastering Olympiad-level physics problems.The conclusion reports gold-medal performance on recent Olympiad competitions.Source: §6 · p. 16
  • The authors attribute P1’s achievements to combining reinforcement-learning post-training with test-time scaling through an agentic framework.Source: §6 · p. 17
  • P1 also performs well on math and coding tasks, and the authors describe Olympiad-level physics mastery as a milestone toward AI systems that may assist or pioneer physics research.Source: §6 · p. 17

7. Acknowlegement

The acknowledgements credit Shanghai AI Laboratory for support and name open-source projects used in the research’s modeling, reinforcement-learning, and inference infrastructure.

7. Acknowlegement

  • The work was supported by Shanghai AI Laboratory, with Qwen3, slime, verl, sglang, and Megatron-LM acknowledged for contributions to the research infrastructure.Source: §7 · p. 17

A.1. Test-time Reinforcement Learning

This section adapts P1-30B-A3B at test time using majority-vote pseudo-labels from unlabeled HiPhO data. It reports answer-level evaluation in Table 5.

A.1. Test-time Reinforcement Learning

  • TTRL adapts P1-30B-A3B using unlabeled HiPhO test data, with majority-vote consensus from 32 sampled responses serving as pseudo-labels.The checkpoint is fine-tuned for 64 steps with GSPO.Source: §A.1 · p. 23
  • Multiple-choice questions are filtered because preliminary experiments found that they inflated consensus ratios and often led to reward hacking.Source: §A.1 · p. 23
  • Table 5 reports HiPhO performance at answer level, using Qwen3-30B-A3B-Instruct as the judge.Source: Table 5 · p. 23

A.2. Case Study

The case study evaluates P1-235B-A22B on an IPhO 2025 problem about optimizing energy dissipation in Cox’s timepiece. The model scores perfectly and is credited with correctly handling the physical constraint, modeling, and optimization.

A.2. Case Study

  • Cox’s timepiece harnesses atmospheric-pressure fluctuations to power a clock through mercury movement between two vessels.The case study frames the task as a multi-step combination of mechanical analysis, constraint formulation, and calculus-based optimization.Source: §A.2 · pp. 23–24
  • The P1-235B-A22B model achieved a perfect score of 1.0/1.0 on IPhO 2025 Question 3-Part 3-C4.Source: §A.2 · pp. 23–24
  • The model is credited with identifying the stop-position force balance, modeling parameter relationships, and applying constrained optimization.Source: §A.2 · p. 23
  • The simplified setup has a mercury cistern and evacuated barometric tube suspended through pulleys, with solid friction and stops limiting displacement.The total mercury volume is 5 L, and displacement is bounded by −X≤x≤X.Source: §A.2 · p. 24
  • The problem asks for optimal friction force and displacement, maximum energy, and a numerical energy value under specified conditions.The stated data include S_c=210 cm^2, A=5 × 10^2 Pa, g=9.8 m·s−2, and mercury density 13534 kg/m3.Source: §A.2 · p. 24

P1 Solution Overview

The solution models frictional energy dissipation per period under simplifying assumptions, then maximizes it subject to the force available from atmospheric-pressure fluctuations. It uses the resulting work function to determine the optimal displacement.

P1 Solution Overview

Loading 2511.13612v1…