DataChef: Cooking Up Optimal Data Recipes for LLM Adaptation via Reinforcement Learning

1 Introduction

• A data recipe is a multi-stage pipeline transforming heterogeneous sources, but recipe design remains human-orchestrated and exhaustive search becomes infeasible as data, models, and operations scale.LLMs automate individual components such as filtering and synthesis, but these commonly use rigid, hand-crafted prompts or algorithms.Source: §1 · pp. 1–2
• The task takes a target benchmark and available data sources as input, then requires a complete, executable pipeline to create adaptation data; curated task data is absent and full training makes online reward costly.The task also requires analyzing heterogeneous sources and applying domain-specific processing operations.Source: §1 · p. 2
• The task pool spans 31 benchmarks, 19 domains, and 257 datasets, with 25 training and 6 held-out tasks; a Data Verifier estimates processed-data utility without model training.The verifier provides a low-latency proxy reward, correlates with downstream performance, and is more robust across tasks than existing data-scoring metrics.Source: §1 · p. 2
• DataChef-32B matches Gemini-3-Pro on six held-out tasks, beats DEITA on most, and its recipes score 66.7 on AIME’25 and 46.3 on ClimaQA, surpassing official Qwen3-1.7B.The reported scores are for recipes adapting Qwen3-1.7B-Base.Source: §1 · p. 2

Data Pipelines

• Prior pipelines rely on expert-designed heuristics or standardized frameworks, while Data-Juicer Sandbox still needs costly downstream-training feedback; DataChef instead generates recipes end-to-end.Examples of the heuristics include data mixing (Liu et al., 2025b), sampling (Xu et al., 2024; Chen et al., 2025d), and synthesis (Chen et al., 2025c).Source: §2 · pp. 2–3

LLM Agents for Data Science

• Existing LLM data-science agents use heuristic workflows, trial-and-error, or external knowledge, but often target curated Kaggle tasks; DataChef handles arbitrary tasks and datasets by generating training recipes.Prompt-driven approaches remain static and constrained by LLM knowledge, while learning-based agents are typically evaluated on static datasets and curated initial code.Source: §2 · p. 3

Data Evaluation

• Data evaluation methods use handcrafted indicators, predictive models, or LLM judges, but their alignment with downstream performance is underexplored; this work compares representative methods across fine-tuning tasks.Prior validation typically compares particular data selections against baselines rather than conducting systematic correlation analysis.Source: §2 · p. 3

3.1 Problem Formulation

• A task is an instruction, evaluation metric, and set of raw sources; a recipe pairs a pipeline with the dataset it produces, and the policy aims to maximize downstream performance.In the experiments, pipelines are implemented as Python scripts.Source: §3.1 · p. 3

3.2 Task Pool Construction

• The pool spans 19 domains and 31 benchmarks, draws on 257 data sources, and yields 25 seed tasks plus 6 held-out tasks; augmentation expands training to 5K unique instances.The held-out set contains 3 in-domain and 3 out-of-domain tasks.Source: §3.2 · p. 4

3.3 End-to-end Data Recipe Generation

• The policy generates a natural-language pipeline plan and executable code; cold-start SFT on demonstrations from separate reasoning and coding models stabilizes subsequent reinforcement learning.Rejection sampling retains demonstrations based on execution success and data quality.Source: §3.3 · p. 4
• The Data Verifier scores sampled dataset instances as 0 for invalid, format-error, or incorrect outputs, 0.4 for task mismatch, and 1.0 for passing outputs.The average score on a random subset estimates dataset quality; empty outputs and training-format violations receive separate penalties.Source: §3.3 · pp. 4–5
• GRPO samples groups of candidate recipes for each training task and optimizes a clipped policy objective using group-relative advantages and KL regularization.The objective uses a fixed reference policy and an importance ratio between current and previous policies.Source: §3.3 · p. 6

4.1 Setups

• Experiments used 3 in-domain and 3 out-of-domain held-out tasks, with 32 candidate recipes per task and downstream fine-tuning of Qwen3-1.7B-Base.The study reports DVSavg@32, the mean verifier score across candidates with invalid recipes scored 0, and DBS from a randomly sampled valid recipe; it also reports the best candidate’s downstream score as an oracle upper bound. DataChef-32B was trained with 5K synthetic instances for cold-start SFT and then GRPO RL; baselines included leading LLMs, manually processed source data, IFD, DEITA, and the official Qwen3-1.7B checkpoint.Source: §4.1 · p. 6

4.2 Main Results

• +8.6% in-domain and +9.2% out-of-domain DVSavg@32, plus +10.7% and +7.4% DBS, were DataChef-32B’s average gains over Qwen3-Next-80B ⊕ Kimi-K2.DataChef-32B also achieved performance comparable to Gemini-3-Pro.Source: §4.2 · p. 6
• Selecting the most promising recipe from 32 samples (Oracle Upper Bound), DataChef-32B outperformed SINGLE-SOURCEbest, IFD, and DEITA across most tasks, achieving a 130.3 average in-domain score.It also scored 66.7 on AIME’25 and 46.3 on ClimaQA, surpassing the official checkpoint.Source: §4.2 · pp. 6–7

4.3 Data Verifier

• Across six held-out tasks, the Data Verifier achieved the highest average Pearson correlation, 0.59, and maintained positive correlations in every task, with 4 of 6 satisfying p < 0.1.Baselines showed cross-domain variance, including −0.48 for IFD on Math and −0.15 for DEITA on Code.Source: §4.3 · pp. 7–8, Figure 3 · p. 7

4.4 Ablation and Analysis

• In-domain and out-of-domain DVSavg@32 improved by 3.6% for the 8B model and 3.7% for the 32B model with RL, which primarily improved out-of-domain generalization while preserving in-domain performance.Reward values rose and their standard deviation decreased during training.Source: Figure 4 · p. 7, §4.4 · p. 8
• Omitting cold-start SFT degraded performance across domains, while direct RL generated simpler pipelines; replacing the fine-grained verifier reward with a constant success reward also caused noticeable drops.The authors hypothesize that without SFT warm-up, reward hacking favors safe, trivial scripts over data-quality optimization; DataChef-8B instead used operations such as filtering and augmentation.Source: Table 2 · p. 7, §4.4 · p. 8
• Pairing Qwen3-32B with Kimi-K2-Instruct as planner and coder raised DVSavg@32 by 18.0% in-domain and 14.5% out-of-domain at inference, but remained suboptimal to end-to-end training.The result suggests integrated training of planning and coding capabilities is important for optimal recipe generation.Source: Table 3 · p. 7, §4.4 · p. 8
• On the out-of-domain financial task, DataChef-32B identified and prioritized high-utility datasets, and its recipes could augment or synthesize task-specific data and extract relevant subsets with generated keywords.The analysis compared recipes from Qwen-32B and DataChef-32B, using downstream performance to categorize high- and low-performing sources.Source: §4.4 · pp. 8–9

5 Conclusion

• DataChef-32B uses a data verifier as a cost-effective reward function for online reinforcement learning.The work also establishes a holistic dataset for training and evaluation.Source: §5 · p. 9
• DataChef-32B demonstrates strong generalization, matching human-level expertise on specific benchmarks.The authors frame the work as bridging data curation and model evolution toward self-evolving AI.Source: §5 · p. 9
• Automated data recipe generation is proposed to streamline LLM adaptation.The paper presents this as a novel paradigm.Source: §5 · p. 9

Limitations

• LLM-as-a-Judge proxy rewards may sacrifice precision on niche tasks, motivating specialized evaluators with higher-resolution reward signals.Source: §5 · p. 9

A.1 Details of Task Pool

• Table 4 lists the benchmarks used in the task pool along with their domains and usage.The supplied table caption identifies its scope but provides no benchmark names or values.Source: §A.1 · p. 13, Table 4 · p. 13
• An LLM generates 3–5 high-relevance keywords tailored to each downstream task to guide dataset retrieval.Source: §A.1 · p. 13
• The generated keywords are used to search both the Hugging Face and Google Search APIs for candidate datasets.Source: §A.1 · p. 13
• Retrieved datasets are ranked by community popularity, such as likes, and the top four candidates are retained for each keyword.Source: §A.1 · p. 13
• A strict verification protocol checks that retrieved candidates do not overlap with the benchmark, preventing data leakage.Source: §A.1 · p. 13

A.2 Prompt Templates and Model Selection

• The Data Verifier is gpt-oss-120b (OpenAI et al., 2025), an MoE model with about 5B active parameters per token for high inference speed.Its rubric-based evaluation prompt is presented in Fig. 7.Source: §A.2 · p. 13
• Cold-start supervision uses Qwen3-Next-80B-A3B-Thinking (Yang et al., 2025) for planning and reasoning and Kimi-K2-Instruct (Team et al., 2025) for code implementation.Their prompts are presented in Figs. 8 and 9, respectively.Source: §A.2 · p. 13

• IFD uses Qwen2.5-3B-Instruct; scores > 1 are treated as outliers and assigned 0 for robust correlation analysis (Li et al., 2024a).Source: §B.1 · p. 13
• DEITA defines the final data score as Complexity Score multiplied by Quality Score, computed with official repository checkpoints (Liu et al., 2024).Source: §B.1 · p. 13
• RewardModelScore uses Skywork-Reward-V2-Llama-3.1-8B-40M to estimate response quality (Liu et al., 2025a).Source: §B.1 · p. 13
• VendiScore uses Qwen3-Embedding-0.6B sample embeddings and Euclidean distance to measure dataset diversity.Source: §B.1 · p. 13

B.2 Evaluation Setup

• OpenCompass evaluations use xVerify-9B-C for Physics average accuracy across sub-tasks and AIME’25 average accuracy over eight responses per question on the 2025 Part I and II subset; LiveCodeBench v6 reports pass@1 with LCBCGenerationEvaluator.Source: §B.2 · pp. 13–14
• ClimaQA uses xVerify-9B-C for average accuracy across sub-tasks; OpenFinData uses OpenFinDataKWEvaluator for the same metric, while CHID reports average accuracy on development and test sets.Source: §B.2 · p. 14

B.3 Oracle Selection

• The oracle filters 32 recipes to the top eight by Data Verifier Score, then a human expert checks format, context integrity, and pipeline comprehensiveness before selecting one for training.Source: §B.3 · p. 14

B.4 Budget

• DBS evaluation uses a 10K-sample budget, randomly downsampling larger recipe outputs and using all samples when fewer are generated; training and DVS evaluation use 100 samples for rapid proxy reward calculation.Source: §B.4 · p. 14

C Case Study

• DataChef-32B’s ClimaQA pipeline uses LLMs to augment data into task-specific formats, synthesize samples, and select relevant subsets with self-generated keywords.Source: C Case Study · p. 14

D Additional Results on Correlation Analysis

• Across all six evaluation tasks, the correlation analysis validates the Data Verifier’s robustness over baseline metrics across diverse domains.The results are presented in Fig. 11.Source: D Additional Results on Correlation… · p. 14

E Computational Cost Analysis

• SFT averages 2.4 minutes per step on 8 H200 GPUs (batch size 32), while each RL step averages 20.2 minutes for 128 candidate recipes using 8 policy GPUs and 2 verifier GPUs.Data Verifier inference accounts for less than 2 minutes of an RL step, demonstrating its computational efficiency.Source: E Computational Cost Analysis · p. 14