Source-linked AI summary
Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, Nick Haber
TL;DR
Existing math datasets trade off small-scale quality against large-scale uncertainty, while RL methods require large corpora of diverse prompts with verifiable answers. Big-Math addresses this gap through human-verified filtering and reformulation, yielding over 250,000 RL-oriented problems and 47,000 additional reformulated questions.
Problem
Existing open math datasets trade off small collections of high-quality problems against larger corpora of uncertain quality, while RL methods require large-scale, diverse prompts with verifiable answers.
Method
Big-Math rigorously filters, cleans, and curates open datasets using human-in-the-loop verification, then reformulates multiple-choice problems into open-ended questions.
Results
Big-Math contains over 250,000 problems, including 47,000 reformulated questions, with analyses covering problem diversity and difficulty.
Takeaways & Limitations
Big-Math bridges math-data quality and quantity with a corpus designed for RL training and intended to support reasoning models across varied capabilities and training requirements.
Takeaways & Limitations
The source-agnostic answer filter cannot guarantee answer correctness or identify all incorrect answers, because model mistakes and difficulty limit its reliability.
Abstract
from arXiv · showhide
Increasing interest in reasoning models has led math to become a prominent testing ground for algorithmic and methodological improvements. However, existing open math datasets either contain a small collection of high-quality, human-written problems or a large corpus of machine-generated problems of uncertain quality, forcing researchers to choose between quality and quantity. In this work, we present Big-Math, a dataset of over 250,000 high-quality math questions with verifiable answers, purposefully made for reinforcement learning (RL). To create Big-Math, we rigorously filter, clean, and curate openly available datasets, extracting questions that satisfy our three desiderata: (1) problems with uniquely verifiable solutions, (2) problems that are open-ended, (3) and problems with a closed-form solution. To ensure the quality of Big-Math, we manually verify each step in our filtering process. Based on the findings from our filtering process, we introduce 47,000 new questions with verified answers, Big-Math-Reformulated: closed-ended questions (i.e. multiple choice questions) that have been reformulated as open-ended questions through a systematic reformulation algorithm. Compared to the most commonly used existing open-source datasets for math reasoning, GSM8k and MATH, Big-Math is an order of magnitude larger, while our rigorous filtering ensures that we maintain the questions most suitable for RL. We also provide a rigorous analysis of the dataset, finding that Big-Math contains a high degree of diversity across problem domains, and incorporates a wide range of problem difficulties, enabling a wide range of downstream uses for models of varying capabilities and training requirements. By bridging the gap between data quality and quantity, Big-Math establish a robust foundation for advancing reasoning in LLMs.
1. Introduction
Big-Math addresses the shortage of large, high-quality math datasets suitable for reinforcement learning by filtering and curating problems with verifiable answers. It also adds reformulated open-ended problems to expand coverage while preserving RL-relevant properties.
- Motivation: RL research lacks large, high-quality math datasets with verifiable answers, leaving researchers to choose between limited human-written data and uncertain machine-generated data.GSM8k and MATH contain 8,000 and 12,000 problems, respectively.
- Dataset criteria: The dataset targets uniquely verifiable solutions, open-ended formulations, and closed-form answers.These criteria support reliable verification, discourage guessing, and enable automated evaluation.
- Construction: A human-in-the-loop filtering process iteratively improves filters through manual verification until each reaches at least 90% precision and recall.The process extracts over 200,000 problems from three openly available datasets before adding reformulated problems.
- Reformulation: Big-Math-Reformulated contributes 47,000 problems by converting multiple-choice questions into open-ended formats while preserving their integrity and complexity.This approach addresses the difficulty of using multiple-choice questions in RL environments.
- Contribution: Big-Math contains over 250,000 math problems designed specifically for reinforcement learning.The dataset combines filtered problems with extracted answers and reformulated questions.
2. Big-Math
Big-Math is constructed by collecting openly available datasets, applying source-specific and source-agnostic cleaning and filtering, and reformulating selected multiple-choice problems into open-ended questions. The resulting dataset prioritizes problems suitable for reinforcement learning while retaining a large scale.
- Dataset Collection: Three established datasets—HARP, Omni-MATH, and NuminaMath—provide the main source material for Big-Math.The collection includes mostly human-written problems and one selected synthetic source.
- Dataset Collection: The combined source collection contains over 640,000 problems before cleaning and filtering.The raw collection includes duplicates, undesirable content, and problems that fail the dataset’s desiderata.
- Source-specific Filtering and Cleaning: Source-specific cleaning removes dataset-dependent artifacts, including figures, attributions, competition metadata, duplicates, and problems without exactly one boxed answer.NuminaMath is deduplicated with MinHashLSH and filtered using boxed-answer extraction; HARP and Omni-MATH receive separate content cleaning.
- Source-specific Filtering and Cleaning: 463,426 problems remain after source-specific filtering, but the collection still contains problems unsuitable for reinforcement learning.Additional source-agnostic filtering is therefore applied across the full collection.
- Source-agnostic Filtering: Model-based solve-rate filtering does not guarantee answer correctness and may remove difficult problems that the evaluating models cannot solve.The authors identify stronger math-specific models as one possible improvement.
- Big-Math-Reformulated: Big-Math-Reformulated adds 47,010 reformulated and filtered problems by converting multiple-choice questions into open-ended questions while preserving their original integrity and complexity.The reformulation process targets high-quality questions otherwise lost because multiple-choice formats can reward incorrect reasoning chains.
3. Analysis and Discussion
Big-Math spans a wide range of problem difficulties and mathematical domains, while its filtering analysis identifies substantial opportunities to recover additional usable questions. The dataset’s difficulty and domain distributions provide guidance for matching subsets to model capability and training needs.
- Dataset Difficulty: 71,926 problems (28.64%) fall in the easiest difficulty quintile, while 91,647 (36.50%) fall in the hardest.The quintiles are defined by solve rates above 80% for the easiest and below 20% for the hardest.
- Dataset Difficulty: 34.44% of Big-Math-Reformulated problems are in the hardest quintile, and 16.42% are in the second hardest, exceeding half of the subset.The reformulated subset is skewed toward more difficult problems than the rest of the dataset.
- Dataset Difficulty: Differential equations, discrete mathematics, and abstract algebra are the most difficult domains, whereas prealgebra is easiest by a wide margin.Most remaining domains show broad difficulty distributions; the relative ease of linear algebra and difficulty of geometry may reflect classification or model-training artifacts.
- Dataset Diversity: The domain distributions have long tails under both ontologies, with Math Word Problems largest under Gao et al. and operations research largest under the MSC classification.Math Word Problems disproportionately originate from Orca-Math, which contains over 66,000 such problems; the dataset also includes applications in electromagnetic theory, thermodynamics, and fluid mechanics.
- Filter Analysis: The multiple-choice filter removes approximately 18% of data, while multi-part and proof filters show substantial disagreements between regular-expression and model-based classifiers.The multi-part filter differs by 14,000 questions for Orca-Math, 6,500 for Olympiads, and 2,700 for MATH; the proof filter removes 10,000 more problems with the model-based approach.
- Filter Analysis: The dataset excludes 625 HARP problems using the Asymptote vector graphics language, leaving their usefulness for image-containing mathematical problems as an open area for study.The paper contrasts this exclusion with evaluation settings containing Asymptote images.
4. Future Directions
The paper identifies future work in scaling RL data, improving filters and verifiers, reducing human oversight, and expanding the methodology beyond mathematics.
- Scaling: Scaling laws for RL training remain an open direction because the efficiency of different training algorithms is unknown.The paper also points to studying how training data and inference-time compute interact.
- Filter and verifier improvements: The filters are overly strict, while improved answer extraction and verifiers could include proofs, multi-part questions, and more complex problems.The current pipeline retains data extractable through simple string matching.
- Model-based filtering: Future model-based filters could use LLM-as-a-judge or related quality checks to reduce reliance on human oversight.The paper proposes exploring RL-trained filtering models and methods such as Meta-RL or continual learning.
- Domain expansion: Much of the filtering methodology could extend beyond mathematics, although some components, such as the proof filter, are domain-specific.The paper identifies multiple-choice and multi-part question filters as examples of more transferable components.
5. Conclusion
The paper concludes that RL reasoning research needs large-scale, diverse prompts with verifiable answers and presents Big-Math to address this data gap. Big-Math combines over 250,000 filtered and reformulated problems, with analyses supporting its diversity and utility for future reasoning research.
- Conclusion: Many RL methods for reasoning models require large-scale, high-quality, diverse prompts with verifiable answers, while similar results across methods suggest data may limit performance.The conclusion frames dataset quality and quantity as central concerns for RL-based reasoning research.
- Conclusion: Over 250,000 Big-Math problems satisfy unique verifiability, open-ended formulation, and closed-form answer requirements through human-in-the-loop filtering.The dataset is designed to bridge data quality and quantity for RL training.
- Conclusion: 47,000 Big-Math-Reformulated problems reclaim data lost during filtering by converting existing closed-ended problems into high-quality problems for the dataset.The reformulated subset is presented as an additional contribution to Big-Math.
- Conclusion: Releasing Big-Math aims to catalyze RL progress and establish a shared benchmark for mathematical reasoning with LLMs.The paper also positions the dataset for studying scaling laws, training dynamics, and new RL algorithms.
A. Filter Statistics
The filter statistics organize Big-Math’s filtering outcomes by subset and distinguish regular-expression filters from Llama-3.1-70B model-based filters.
- A. Filter Statistics: The filter-statistics display reports results across multiple-choice, true/false, yes/no, multi-part, proof, semantic, total, and deduplication filters.
- A. Filter Statistics: Rows identify the filtered subsets, while columns are divided between regular-expression and Llama-3.1-70B filtering results.The caption labels regular-expression columns as (R) and model-based columns as (L).
B. Domains By Source
The domain-by-source analysis presents mathematics-domain distributions using the Gao et al. ontology and organizes the breakdown by dataset source.
- B. Domains By Source: The domain display includes categories such as abstract algebra, calculus, differential equations, discrete mathematics, geometry, statistics, and other mathematics areas.
- B. Domains By Source: The source breakdown associates problems with domains including algebra, equations, mathematics, algebra problems, and theory.
- B. Domains By Source: The accompanying caption states that the display breaks down problems by source and domains defined by Gao et al. (2024).
C. Reformulation Examples
Figure 9 illustrates reformulation from an original multiple-choice problem to an open-ended question, followed by critical evaluation using successful and failed examples.
- The successful example begins with a multiple-choice question asking for the sum of the prime factors of 2010.
- The reformulation process extracts key information from the original problem before generating an open-ended version.The pipeline then critically evaluates whether the reformulation succeeded.
- Figure 9 contrasts successful and failed reformulations to show how the evaluation stage distinguishes acceptable transformations.
D. Prompts
The appendix prompts classify mathematical questions and guide reformulation toward single-answer, open-ended problems while illustrating diverse source examples and evaluation criteria.
- D.1. Llama-3.1-70B Filter prompts: The D.1 prompt classifies whether a question is multiple choice by detecting explicit answer options.It requests only a yes-or-no classification.
- D.2. Reformulation Prompts: The D.2 prompts classify proof, yes/no, true/false, and multi-part questions before reformulation.They distinguish single-answer questions from questions requiring separate answers.
- D.2. Reformulation Prompts: Additional examples cover polar curves, absolute-value functions, ellipses, polygonal numbers, algebraic expressions, inequalities, and geometric solids.These examples demonstrate varied mathematical formats and domains used in the prompt set.
- D.2. Reformulation Prompts: A highlighted worked example concludes with the circle equation (x−2)^2 + (y−5)^2 = 16 as the required full answer.