PaperlayerView exact revision on arXiv

Source-linked AI summary

VOYAGER: A Training Free Approach for Generating Diverse Datasets using LLMs

Avinash Amballa, Yashas Malur Saidutta, Chi-Heng Lin, Vivek Kulkarni, Srinivas Chappidi

arXiv:2512.12072v2cs.CLcs.LG

TL;DR

Synthetic data from post-trained LLMs often lacks diversity, while existing approaches have constraints in scalability, computation, or model access. VOYAGER iteratively optimizes a volume-based diversity measure using determinantal point processes and prompt refinement. It improves diversity by 1.5−3x over baselines and significantly outperforms methods across creative-writing and reasoning tasks.

  • Problem

    Post-trained LLMs often generate synthetic data with poor diversity, while reward-based approaches are computationally expensive and restricted to open-weight models.

    Source: §1 · p. 1

  • Method

    VOYAGER iteratively uses determinantal point processes and prompt refinement to approximately maximize a volume-based measure of dataset diversity.

    Source: §4.2 · p. 4, §6 · p. 9

  • Results

    VOYAGER improves diversity by 1.5−3x over baseline methods and significantly outperforms methods on creative-writing and reasoning tasks.

    Source: §5.4 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    VOYAGER is a training-free, scalable approach that requires no model-weight access and is applicable to closed-weight models.

    Source: §1 · p. 1, §6 · p. 9

    The work focuses on text generation and does not consider multimodal data; it also depends on high-quality instruction-following LLMs and robust embedding models.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large language models (LLMs) are increasingly being used to generate synthetic datasets for the evaluation and training of downstream models. However, prior work has noted that such generated data lacks diversity. In this paper, we propose Voyager, a novel principled approach to generate diverse datasets. Our approach is iterative and directly optimizes a mathematical quantity that optimizes the diversity of the dataset using the machinery of determinantal point processes. Furthermore, our approach is training-free, applicable to closed-source models, and scalable. In addition to providing theoretical justification for the working of our method, we also demonstrate through comprehensive experiments that Voyager significantly outperforms popular baseline approaches by providing a 1.5-3 times improvement in diversity.

1 Introduction

Synthetic data is valuable where real-world data is scarce, but post-trained LLMs often generate insufficiently diverse data. VOYAGER addresses this with a training-free, theory-grounded framework that improves diversity over baselines.

1 Introduction

  • Post-trained LLMs often generate synthetic data with poor diversity, while sampling can miss semantic clustering and prompt-based control is not scalable.Post-training reward approaches are computationally expensive and restricted to open-weight models.Source: §1 · p. 1
  • VOYAGER generates diverse synthetic data without training or access to model parameters, making it compatible with closed-weight models.It maintains and dynamically prunes a fixed-size anchor set of representative regions.Source: §1 · p. 1
  • VOYAGER uses determinantal point processes to optimize diversity as geometric volume and improves diversity by 1.5−3 times over baselines.The paper also reports that empirical evaluations consistently outperform strong baselines across a suite of tasks.Source: §1 · p. 1

2 Related Work

Prior diversity-promotion methods either manipulate generation, guide prompts, or update model weights. VOYAGER instead uses text-based gradients to iteratively generate a globally diverse dataset.

2 Related Work

  • Existing diversity-promotion methods are grouped into training-free and training-based approaches.Training-free methods use decoding strategies or prompt engineering without updating model weights.Source: §2 · p. 2, §2.1 · p. 2

2.1 Training-free Methods

  • Sampling and decoding methods manipulate output probabilities but require raw logits and can produce incoherent or grammatically flawed outputs.VOYAGER instead optimizes a global diversity proxy without model weights or logits.Source: §2.1 · p. 2
  • Prompt-diversity methods guide topics or styles but depend on predefined granular topics and may yield topical rather than semantic diversity.Distinct topics can still share substantial vocabulary.Source: §2.1 · p. 2

2.2 Training-based Methods

  • Training-based approaches optimize diversity rewards by fine-tuning weights, incurring computational overhead and excluding closed-source models.Prior work also measures dataset diversity by volume or integrates a determinant metric into post-training; VOYAGER instead iteratively generates globally diverse data using text-based gradients.Source: §2.2 · p. 2

3 Background

VOYAGER combines determinantal point processes with prompt refinement using textual gradients. These provide a diversity-oriented subset-selection mechanism and a text-space procedure for improving prompts.

Background

  • A determinantal point process assigns subset probability in proportion to the determinant of the subset's kernel matrix.The determinant measures the volume spanned by items, so higher values correspond to more diverse, less redundant subsets.Source: §3 · p. 2
  • VOYAGER relies on both determinantal point processes and textual-gradient prompt refinement.The former supports subset selection; the latter refines prompts toward a reward objective.Source: §3 · p. 2
  • Textual-gradient prompt refinement asks an LLM-judge to suggest prompt changes, then has another LLM incorporate those suggestions.The procedure operates in text space and aims to improve a reward metric.Source: §3 · p. 2

4 VOYAGER – Algorithm

VOYAGER iteratively builds a dataset by exploring with prompts, accepting samples that increase anchor-set volume, and refining explorers. Its DPP-based pruning approximates a high-volume objective while limiting computation through fixed-size sets.

4 VOYAGER – Algorithm

  • VOYAGER seeks to approximately maximize the determinant of a dataset's similarity matrix, using it as a measure of diversity.The determinant corresponds to squared volume.Source: §4 · p. 2, §4.2 · p. 4

4.1 Overview

  • The algorithm maintains fixed-size explorer and anchor sets, adds generated batches to the dataset, and uses DPP sampling to select diverse anchors and successor explorers.The anchor set represents diverse regions of the underlying dataset.Source: §4.1 · p. 3

4.1.1 Exploration Step

  • Exploration generates a batch, accepts instances whose marginal volume gain exceeds threshold τ, and refines explorers when instances are rejected.Marginal gain is computed against the fixed-size anchor set rather than the growing full dataset to reduce computational expense.Source: §4.1.1 · p. 3

4.2 Theoretical Justification

  • The maximum-volume submatrix is a proxy for maximizing the similarity matrix determinant because it is a quasi-best low-rank approximation.The exact subset problem is NP-hard; DPP samples favor high-volume subsets without finding the exact solution.Source: §4.2 · p. 4
  • VOYAGER constructs and prunes a high-volume anchor set, which the paper links to a high maximum-volume submatrix, effective rank, and dataset diversity.The anchor set is pruned to size k using a k-DPP that favors high-volume subsets.Source: §4.2 · pp. 4–5

4.3 Computational Efficiency

  • VOYAGER's computational cost depends chiefly on maximum anchor-set size, candidate-beam size, batch size, and iteration count.The paper says it is generally more efficient than sampling a DPP over a universe larger than the requested dataset, which would cost O(|l|^3).Source: §4.3 · p. 5

5 Experiments

These experiments evaluate VOYAGER across creative-writing and reasoning generation tasks, using several diversity measures alongside quality and call-count assessments. The authors report higher diversity than baselines, examine human ratings and ablations, and test whether generated data improves model training.

5.1 Generation Tasks and Evaluation Metrics

  • Experiments covered topical sentences, conversations, poems, movie plots, grade-school math questions, and logical-reasoning questions, assessing semantic, lexical, and broader dataset diversity.Diversity measures included mean cosine distance, mean Jaccard distance, and Vendi score; the evaluation also tracked generation quality and LLM calls.Source: §5.1 · pp. 5–6

5.4 Results and Discussion

  • Vendi scores improved 2.96 times over DEFAULT and 0.43 times over HIERARCHICAL across creative tasks, and 4.12 times and 1.02 times, respectively, across reasoning tasks, without significant quality degradation.The reasoning-task results also report that VOYAGER significantly outperforms all baselines.Source: Table 1 · p. 6, Table 2 · p. 6, §5.4 · p. 7

5.2 Baseline Methods; 5.3 Experimental Settings

  • Baselines included default generation, temperature sampling, diverse instructions, history conditioning, hierarchical prompting, and DPP subset selection; experiments used GPT-4o mini and text-embedding-3-small.The similarity kernel combined RBF embeddings and Jaccard similarity with weights 0.7 and 0.3.Source: §5.2 · p. 6, §5.3 · p. 7

5.4 Results and Discussion

  • Temperature improved diversity without significant quality loss; DIVERSE-KEYWORD and hierarchical prompting improved it more, history only marginally, while human ratings favored VOYAGER over DEFAULT with good inter-annotator reliability.Human ratings covered Sports and Math tasks.Source: §5.4 · pp. 7–8

5.5 Ablation Studies

  • Random explorer selection slightly reduced diversity and increased LLM calls; disabling textual gradients raised rejection rates and iterations.The textual-gradient comparison disabled prompt refinement against a setting with refinement and beam size one.Source: Table 4 · p. 8, §5.5 · p. 8, Figure 2 · p. 8
  • VOYAGER outperformed other methods on poem diversity even when average output token counts were very similar, indicating its gains did not stem from length alone.The length-controlled task prompt restricted poem outputs to 150 tokens.Source: Table 5 · p. 9, §5.5 · p. 9

5.6 Effectiveness of Data for Synthetic Training Data Generation

  • On GSM8K, Gemma-7b-it trained using 1,000 VOYAGER-generated examples scored 45.7 versus 35.7 for DEFAULT-generated data; 500 VOYAGER examples almost matched or outperformed the DEFAULT-data models.The models were evaluated on the standard GSM8K test set.Source: §5.6 · p. 9, Table 6 · p. 9

6 Conclusion

VOYAGER uses an iterative DPP-based approach to approximately maximize dataset diversity, improving generated-data diversity by 1.5–3x over baseline methods while remaining training-free and scalable.

6 Conclusion

  • VOYAGER is presented as a training-free approach for using LLMs to generate more diverse data.The conclusion characterizes the method as scalable compared with baseline methods.Source: §6 · p. 9
  • 1.5–3x improvement in generated-data diversity over baseline methods is reported for VOYAGER.The method is described as training-free and scalable.Source: §6 · p. 9
  • VOYAGER uses the determinant of a dataset’s similarity matrix, related to the volume spanned by its data, as a diversity measure to approximately maximize iteratively.The approach leverages determinantal point processes and prompt refinement using textual gradients.Source: §6 · p. 9

Limitations

VOYAGER depends on high-quality instruction-following LLMs and a robust embedding model; the paper also identifies limits in its diversity analysis and text-only scope.

Limitations

  • VOYAGER depends on high-quality instruction-following LLMs, especially for textual-gradient extraction as well as sample generation.The LLM must analyze a prompt and its generation, then suggest improvements.Source: Limitations · p. 9
  • Similarity computations require a robust embedding model that can distinguish textual differences in embedding space.Source: Limitations · p. 9
  • The paper says a comprehensive analysis of the types of diversity introduced by VOYAGER is needed to understand its impact and effectiveness fully.Source: Limitations · p. 9
  • The study covers diversity in LLM text generation, not multimodal data, where measuring similarity across modalities poses additional challenges.Source: Limitations · p. 9

A Appendix

The appendix documents Voyager’s sampling and threshold procedures, effective-rank approximation, qualitative examples, and prompt-refinement templates. It also provides evaluation criteria and domain-specific prompts for generated poems, movie plots, grade-school math questions, and logic puzzles.

1.1 VOYAGER- Algorithm

  • Voyager samples subsets with a K-DPP and computes an item’s marginal gain from the change in similarity-kernel volume.The sampling procedure uses the K-DPP method of Kulesza et al. (2012).Source: §1.1 · p. 12

1.2 Recommendation for setting τ

  • The recommended initial threshold uses 100 model outputs, samples 10 with a DPP, and clips a scaled determinant between threshold bounds.The appendix notes that the authors did not invest effort in tuning parameters such as τ.Source: §1.2 · p. 12

1.4 Training details

  • Gemma training on Grade School Math uses batch size 2, LoRA rank 16, four projection target modules, and four gradient-accumulation steps.The target modules are q-proj, v-proj, k-proj, and o-proj.Source: §1.4 · p. 13

1.3 Proof of Lemma 1

  • The effective-rank approximation is tighter for eigenvalue distributions closer to uniform and generally underestimates effective rank as condition numbers increase.The appendix treats the approximation as a useful lower-bound floor that Voyager seeks to raise iteratively.Source: Figure 3 · p. 13, §1.3 · p. 13

1.5 Qualitative results

1.6 Refinement Prompts

  • The refinement templates ask an LLM to diagnose low-diversity outputs and generate concise, positive-language prompt revisions from gradient feedback.The generated revisions address individual suggestions and are projected for relevance to the system prompt.Source: §1.6 · p. 16

1.7 Quality Evaluation Prompts

  • Evaluation prompts assess sports sentences, political conversations, and poems using task-specific criteria including sports content, conversation dynamics, and poetic structure.The criteria also include creativity for sports and politics, and language and style for poems.Source: §1.7 · pp. 17–19
  • The poem rubric scores relevance, creativity, structure, language, and emotional impact from 0 to 5, with distinct low and high descriptors listed for each criterion.Source: §1.7 · p. 19
  • Movie-plot evaluations assess prompt relevance, plot structure, originality, engagement, and creativity, with each criterion scored out of 5.The criteria check for a narrative arc, a fresh concept, and a compelling hook.Source: §1.7 · p. 20
  • Grade-school math-question evaluations score relevance, clarity, and appropriateness of mathematical content, each out of 5.The prompt requests detailed analysis and justifications, plus an overall score.Source: §1.7 · p. 21
  • Grade-school logic-puzzle evaluations score relevance, clarity, and appropriateness of logical content, each out of 5.The prompt requests detailed analysis and justifications, plus an overall score.Source: §1.7 · p. 22
Loading 2512.12072v2…