PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Language Models as Synthetic Data Generators

Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, Graham Neubig

arXiv:2412.03679v2cs.CL

TL;DR

Prior work has developed synthetic-data generation methods, but differing experimental settings hinder systematic comparisons of language models as data generators. AGORABENCH standardizes comparisons across generation tasks and finds that model strengths vary by task, without strong correlation to problem-solving ability.

  • Problem

    Prior studies use heterogeneous generators, training-data quantities, base models, and benchmarks, making it difficult to isolate language models’ data-generation capabilities.

    Source: §1 · p. 1

  • Method

    AGORABENCH compares six language models across nine controlled combinations of three domains and three generation methods, training and evaluating a student model on each synthetic dataset.

    Source: §1 · pp. 1–2

  • Results

    GPT-4o achieved the highest PGR in five of nine settings, while data-generation ability showed no strong correlation with problem-solving ability.

    Source: §1 · p. 2, §4 · p. 5

  • Takeaways & Limitations

    AGORABENCH can help researchers identify data-generator capabilities and let practitioners compare custom generation pipelines against baseline settings.

    Source: §6 · p. 9

    The evaluation uses Llama-3.1-8B as its base model, so generalizability across other base models remains to be investigated.

    Source: Limitations & Potential Risks · p. 9

Abstract

from arXiv · show

Given the increasing use of synthetic data in language model (LM) post-training, an LM's ability to generate high-quality data has become nearly as crucial as its ability to solve problems directly. While prior works have focused on developing effective data generation methods, they lack systematic comparison of different LMs as data generators in a unified setting. To address this gap, we propose AgoraBench, a benchmark that provides standardized settings and metrics to evaluate LMs' data generation abilities. Through synthesizing 1.26 million training instances using 6 LMs and training 99 student models, we uncover key insights about LMs' data generation capabilities. First, we observe that LMs exhibit distinct strengths. For instance, GPT-4o excels at generating new problems, while Claude-3.5-Sonnet performs better at enhancing existing ones. Furthermore, our analysis reveals that an LM's data generation ability doesn't necessarily correlate with its problem-solving ability. Instead, multiple intrinsic features of data quality-including response quality, perplexity, and instruction difficulty-collectively serve as better indicators. Finally, we demonstrate that strategic choices in output format and cost-conscious model selection significantly impact data generation effectiveness.

1 Introduction

Synthetic data offers a scalable complement to human labeling, but heterogeneous prior evaluation settings make it difficult to isolate language models’ data-generation capabilities. AGORABENCH controls those settings to compare models and finds task-specific strengths that do not track problem-solving ability alone.

1 Introduction

  • Prior studies varied generators, synthetic-data quantities, student models, and evaluation benchmarks, making data-generation comparisons difficult to isolate.Synthetic data generation is presented as a scalable complement to manual annotation.Source: §1 · p. 1
  • AGORABENCH compares three generation methods across math, instruction-following, and code while controlling prompts, seed data, and training-instance counts.Llama-3.1-8B is trained on each synthetic dataset and evaluated on fixed benchmarks; PGR measures the student model’s relative improvement over its base model.Source: §1 · pp. 1–2
  • GPT-4o leads instance generation (+ 46.75%), while Claude-3.5-Sonnet leads quality enhancement (+ 17.89%).GPT-4o exceeds Claude-3.5-Sonnet (+ 24.14%) and Llama-3.1-405B-Instruct (+ 10.10%) in generating new instances; Claude-3.5-Sonnet exceeds GPT-4o (+ 6.69%) and GPT-4o-mini (+ 5.49%) at refining existing instances.Source: §1 · p. 2
  • Unlike contemporaneous Xu et al. (2024c), AGORABENCH covers three generation settings and analyzes intrinsic data quality in relation to PGR.Source: §1 · p. 2
  • Problem-solving ability does not strongly correlate with data-generation ability; instruction difficulty, response quality, and response perplexity collectively influence student improvement.The top five principal components of intrinsic measurements explain 93.4% of the variance in PGR values.Source: §1 · p. 2

2 Preliminaries: Measuring Data Generation Capabilities of LMs

AGORABENCH organizes data generation into three methods and evaluates its downstream value with Performance Gap Recovered (PGR). PGR complements intrinsic data-quality measures by directly assessing benchmark improvement from training on generated data.

Data generation methods

  • AGORABENCH groups data generation into instance generation, response generation, and quality enhancement.Instance generation expands a small seed set into new instruction–response pairs; response generation supplies answers to fixed instructions; quality enhancement refines existing instructions and/or responses.Source: §2 · p. 3, Figure 2 · p. 3

Performance Gap Recovered

  • PGR measures a data-trained student’s benchmark improvement relative to a reference model sharing its pretrained base.A PGR of 50% means recovering half the reference model’s improvement; values above 100% exceed it, while negative values indicate degraded performance relative to few-shot prompting the base model.Source: §2 · p. 3, Figure 3 · p. 3

PGR vs. intrinsic metrics

  • Unlike intrinsic quality measures, PGR directly evaluates generated data by its impact on student-model performance and complements rather than replaces those measures.The paper identifies response quality, instruction difficulty, and instance diversity as intrinsic measures that do not directly capture this downstream impact.Source: §2 · p. 4

Student-model training

  • The student-model setup uses supervised fine-tuning on unfiltered generated data, without other post-training methods, to assess raw generation capability.Each setting generates 10K instances with the same meta-prompt and seed data; in instance generation, the seed data is also used for training.Source: §2 · p. 4, Table 1 · p. 4

3 Experimental Setting of AGORABENCH

AGORABENCH crosses three domains with three generation methods, controlling the generator comparison within each setting. The experiment specifies domain-specific seed data and benchmarks, and trains each student on data from a single domain.

Domains

  • The benchmark covers math, code, and instruction following across three generation methods, yielding nine settings with 10K generated instances each.Within a setting, generators use the same meta-prompt and seed dataset.Source: §3 · p. 4
  • Students train on one domain at a time to isolate generated-data quality from possible positive or negative cross-domain transfer.The authors note that code training could, for example, improve math performance.Source: §3 · p. 4

Seed datasets

  • Instance-generation seeds are high-quality human-crafted datasets, with 14,856 math, 874 code, and 503 instruction-following instances after excluding examples over 4,096 tokens.The selected sources are GSM8K and MATH for math, MBPP and xP3x for code, and LIMA for instruction following.Source: §3 · p. 4
  • Response-generation settings use 10K Magpie instructions for instruction following and 10K Magpie-Reasoning instances for math and code.This setup models generators attaching responses to a fixed instruction set.Source: §3 · pp. 4–5
  • Quality-enhancement settings use 10K WebInstruct instances for math and instruction following, and CoNaLa examples for code.WebInstruct lacks domain labels, so GPT-4o-mini prepares a separate seed dataset; CoNaLa pairs simple instructions with short code snippets.Source: §3 · p. 5

Benchmarks

  • Student performance is evaluated on GSM8K and MATH for math, MBPP and HumanEval for code, and AlpacaEval-2.0 and Arena-Hard for instruction following.The paper uses test subsets or test sets for the task benchmarks.Source: §3 · p. 5

4 Experimental Results of AGORABENCH

AGORABENCH results show that data-generation performance varies by model and generation setting: GPT-4o leads in new-instance generation, while Claude-3.5-Sonnet excels at quality enhancement. Stronger problem-solving ability or higher cost does not guarantee better data generation.

4 Experimental Results of AGORABENCH

  • GPT-4o leads in instance generation across math (20.6%), code (73.6%), and instruction following (46.1%), with a 46.8% total average PGR.It also achieves the highest average PGR in response generation (35.2%).Source: §4 · p. 5
  • Claude-3.5-Sonnet leads quality enhancement in code (21.8%) and instruction following (17.9%), with a 17.9% total average PGR.It also records the highest code response-generation PGR at 44.5%.Source: §4 · p. 5
  • Llama-3.1-8B-Instruct outperforms the 70B and 405B versions while costing 6 to 32.5 times less; GPT-4o also outperforms Claude-3.5-Sonnet at 1.2 to 1.5 times lower cost.The comparison covers API costs, problem-solving scores, and average AGORABENCH performance.Source: Table 3 · p. 6, §4 · p. 6
  • In code instance generation, Llama-3.1-70B-Instruct (58.7%) and 8B-Instruct (55.7%) outperform Claude-3.5-Sonnet (23.4%) and 405B-Instruct (12.6%).The weaker-performing generators score higher on code problem-solving benchmarks, underscoring that solver strength does not ensure stronger data generation.Source: §4 · pp. 5–6
  • In code quality enhancement, GPT-4o (-8.8%) and GPT-4o-mini (-11.2%) perform worse than other LMs.Source: §4 · p. 6

5 What makes an effective data generator?

This section examines whether problem-solving ability predicts data-generation performance and whether intrinsic properties of generated data help explain student-model improvement.

5.2 Can we predict the student model’s improvement by looking into the data?

  • The top five principal components explain approximately 93.4% of AGORABENCH-result variance, with PC-1 shaped by instruction difficulty and diversity, PC-2 by response quality and difficulty, and PC-3 by diversity, response quality, and solver ability.The five components explain 39.2%, 30.4%, 11.9%, 7.0%, and 4.9%, respectively.Source: Figure 5 · p. 7, §5.2 · p. 8

5.1 Is the best solver necessarily the best generator?

  • Figure 4 shows no strong linear correlation between problem-solving benchmark scores and AGORABENCH PGR scores at either granularity level, suggesting traditional benchmark performance may not predict effectiveness as a data generator.Source: §5.1 · p. 7

5.2 Can we predict the student model’s improvement by looking into the data?

  • Regression on weighted top-five principal components yields R2 = 0.325 with p < 0.001, improving on solver scores alone, though the moderate fit suggests additional intrinsic metrics may be needed.Source: Figure 6 · p. 8, §5.2 · p. 8
  • The intrinsic evaluation considers instruction difficulty, response quality, response perplexity, and instruction and response diversity; PCA is used because these metrics are interdependent.Source: §5.2 · pp. 7–8
  • Intrinsic metrics contribute similarly to PGR variance, with average loading strengths from 0.189 to 0.256; response-quality metrics contribute slightly more than diversity or instruction-difficulty metrics.Source: Table 4 · p. 8, §5.2 · p. 8

6 Conclusion

AGORABENCH standardizes evaluation of language models as data generators. The paper presents it as a basis for studying generator capabilities and comparing practical data-generation pipelines.

6 Conclusion

  • AGORABENCH systematically evaluates language-model data-generation capabilities using standardized settings and metrics.Source: §6 · p. 9
  • The authors propose using the benchmark to identify capabilities of effective generators and potentially develop models specialized for data generation.Source: §6 · p. 9
  • Practitioners can compare custom generation methods, seed datasets, or meta-prompts against AGORABENCH baseline settings.Source: §6 · p. 9

Limitations & Potential Risks

The study’s scope is constrained by its student-model and data-volume settings, while broader risks of synthetic data and proprietary generators remain open concerns. Related work includes multiple data-generation approaches and a contemporaneous comparison limited to response generation.

Limitations & Potential Risks

  • The benchmark does not test base models beyond Llama-3.1-8B or assess whether findings hold when generating more than 10K instances.Preliminary scaling experiments with selected cost-effective LMs generated up to 50K instances; whether generator rankings persist at larger scales remains an open question.Source: Limitations & Potential Risks · p. 9
  • Response perplexity varies inherently with the base-model architecture, so its behavior across other base models remains an open generalizability question.The authors identify Llama-3.1-8B as a widely adopted base model, making findings especially relevant to researchers using similar architectures.Source: Limitations & Potential Risks · p. 9
  • The study focuses on math, coding, and instruction-following performance, leaving synthetic data’s effects on alignment, bias, safety, and robustness for future study.The authors also flag bias, transparency, and permission concerns when proprietary models generate data.Source: Limitations & Potential Risks · p. 9

A Related Work

  • Earlier studies showed that LMs could generate novel instances for post-training, alongside a continuing role for human annotation.The paper situates its work among synthetic-data approaches such as Wang et al. (2023) and Honovich et al. (2023b).Source: A Related Work · p. 13
  • Earlier work developed synthetic-data methods and prompts, including Evol-Instruct [Xu et al. (2024a)] and Magpie [Xu et al. (2024b)].Evol-Instruct prompts an LM to make an existing problem more complex; Magpie extracts instructions from an empty chat template before generating responses.Source: A Related Work · p. 13
  • Xu et al. (2024c) measured LMs’ data-generation capabilities but examined only response generation, whereas this study also tests instance generation and quality enhancement.The authors also conduct analyses of how intrinsic data quality relates to PGR.Source: A Related Work · p. 13

B Further Analysis Experiments

The further analyses compare data quantity with generator cost and examine how meta-prompt design and output format affect performance. Larger datasets from affordable models can outperform smaller GPT-4o datasets in some domains, while the benchmark prompt and free-form output perform better in most tested settings.

B Further Analysis Experiments

  • The quantity-versus-quality analysis asks whether more instances from cheaper models outperform fewer instances from expensive models.The prompt analysis also examines structured JSON versus free-form output and meta-prompt design.Source: §B.1 · p. 13, B Further Analysis Experiments · p. 13

B.2 Effect of Meta-prompts

  • The prompt comparison uses four additional meta-prompts, including an unoptimized version and a JSON-format version for instance generation and quality enhancement.The unoptimized prompt was created in under 10 minutes, unlike the main-experiment prompts developed through more than two hours of iteration.Source: §B.2 · p. 13

B.1 Quantity or quality?

  • 17 times cheaper, GPT-4o-mini’s 50K instances cost 3.4 times less than GPT-4o’s 10K and achieve higher PGR in instruction following and math.In code, Llama-3.1-8B-Instruct showed a similar pattern; the comparison also suggests instruction or response diversity may affect PGR across dataset sizes.Source: §B.1 · p. 13, Figure 7 · p. 14

B.2 Effect of Meta-prompts

  • 4.45% higher performance came from AGORABENCH free-form meta-prompts than JSON-format prompts; the AGORABENCH prompt scored highest in five of six settings.AGORABENCH meta-prompts also exceeded unoptimized versions by a 3.97% average performance gap.Source: §B.2 · pp. 13–14, Table 5 · p. 14

C Response Generation Seed Dataset Construction

The section describes seed-data preparation for domain-specific experiments, including classifying WebInstruct instances for quality enhancement. It also points to the inference hyper-parameters used.

C Response Generation Seed Dataset Construction

  • Separate seed datasets are prepared for instruction following, math, and code to prevent positive or negative transfer from obscuring how synthetic-data quality relates to PGR.Source: C Response Generation Seed Dataset… · p. 14
  • Because WebInstruct lacks math-problem labels, GPT-4o-mini-2024-07-18 classifies its instances for the math and instruction-following quality-enhancement settings.The classification prompt asks whether an instruction-response instance concerns a math-related task or another task and requires a “1” or “2” answer.Source: C Response Generation Seed Dataset… · pp. 14–15
  • Appendix E lists inference hyper-parameters in Table 6.Source: Table 6 · p. 15
  • The classification prompt presents the instance as an instruction and response, using the input and output as placeholders.Source: C Response Generation Seed Dataset… · p. 15

D Problem Solving Abilities of LMs evaluated as Data Generators

The section points readers to evaluation results for six language models and specifies that the experiments use settings from Appendix E.

D Problem Solving Abilities of LMs evaluated as Data Generators

E Details for Training and Evaluating Student Models

Training hyperparameters for student models and hyperparameters for evaluating student models and data-generator LMs are listed in Table 6; evaluation uses specified benchmark libraries and scripts.

E Details for Training and Evaluating Student Models

F Intrinsic Evaluation of AGORABENCH

This appendix reports intrinsic evaluation results, provides the meta-prompts, and describes the prompts and score rubrics used to assess response quality and instruction difficulty.

F Intrinsic Evaluation of AGORABENCH

G AGORABENCH Meta-prompts

H Prompt for Intrinsic Evaluation

  • The intrinsic-evaluation prompts assess response quality and instruction difficulty with GPT-4o and Prometheus-2-8x7B, using rubrics for instruction-following, math, and code.Source: H Prompt for Intrinsic Evaluation · p. 15
  • Math-response and instruction-following quality rubrics assess correctness, reasoning, clarity, precision, helpfulness, relevance, accuracy, depth, creativity, and detail.Source: H Prompt for Intrinsic Evaluation · p. 16
  • Difficulty rubrics rate instruction-following, math, and code problems from 1 to 5, with higher scores describing greater complexity and reasoning demands.The supplied rubric descriptions distinguish simple or direct tasks from advanced or exceptionally complex ones.Source: H Prompt for Intrinsic Evaluation · pp. 17–18
Loading 2412.03679v2…