Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity

• The work argues that inference-time prompting can mitigate mode collapse, preserve aligned models’ inherent diversity, and improve the quality-diversity trade-off without compromising factual accuracy or safety.Mode collapse narrows responses and limits applications including creative writing, social simulation, pluralistic alignment, and synthetic data generation.Source: §1 · pp. 1–2
• Typicality bias—the preference for familiar, fluent, and predictable text—is identified and theoretically and empirically verified as a fundamental data-level cause of mode collapse.This bias may drive collapse even with perfect reward modeling and optimization, affecting alignment methods that rely on reward models.Source: §1 · pp. 1–2
• Verbalized Sampling asks models to generate multiple responses with corresponding probabilities instead of producing a single instance, providing a training-free way to bypass mode collapse.The method is motivated by the observation that distribution prompts recover diversity learned during pretraining, whereas direct prompts tend toward stereotypical modal responses.Source: §1 · pp. 1–2
• VS improves creative writing, social dialogue simulation, synthetic data generation, and open-ended QA by producing more diverse, human-like, useful, and realistic response distributions.In social dialogue simulation, some models perform on par with a dedicated fine-tuned model, while synthetic data improves downstream math performance.Source: §1 · p. 2
• 1.6-2.1× greater diversity over direct prompting, 25.7% higher human evaluation scores, and 66.8% recovery of base-model diversity demonstrate VS’s empirical gains.The paper also observes that more capable models benefit more from VS.Source: §1 · p. 2

Mode Collapse and Alignment

• Aligned models exhibit substantial output-diversity and creative-capacity losses, while prior explanations emphasize algorithmic limitations and majority-style preference amplification.SFT overfitting, cross-entropy training, and rigid chat templates can further restrict diversity before alignment.Source: §2 · pp. 2–3
• Typicality bias provides a complementary data-level explanation for mode collapse, exacerbating the algorithmic causes identified in prior work.This perspective addresses limitations of relying on a single reward model to represent diverse human preferences.Source: §2 · pp. 2–3

Methods to Improve Diversity

• Existing diversity interventions span training, decoding, and prompting, but training and decoding methods can be costly or limited to open-source models, while prompts are often handcrafted.Verbalized sampling is presented as training-free, principled, and broadly applicable.Source: §2 · p. 3
• Earlier response-list and knowledge-verbalization methods lacked theoretical grounding, whereas this work argues that distribution-level queries improve diversity and enable output-diversity tuning.These prior methods addressed question answering, commonsense reasoning, survey simulation, and synthetic data generation (Tian et al., 2023; Xiong et al., 2024; Zhang et al., 2024a; Meister et al., 2024; Wang et al., 2023; Si et al., 2024).Source: §2 · p. 3

3.1. Typicality Bias: Cognitive & Empirical Evidence

• Cognitive mechanisms including mere exposure (Zajonc, 1968; Bornstein, 1989), availability, processing fluency, and schema congruity motivate the typicality-bias hypothesis.These tendencies favor familiar, easily recalled, fluent, and schema-consistent content independently of task-specific quality.Source: §3.1 · p. 3
• Controlling for token count, readability, type–token ratio, and sentence length attenuates but does not eliminate the positive, statistically significant typicality coefficient, making base-model log-probability an imperfect proxy.The paper therefore treats log-probability as a useful typicality-like preference signal rather than a pure psychological measurement.Source: §3.1 · p. 3
• The paper defines response typicality as base-model log-probability, finding that human preferences for higher-typicality responses exceed chance at 51.6–60.8% on human-only HelpSteer data.This proxy was evaluated with five base models across four preference datasets.Source: §3.1 · p. 3
• Human raters significantly preferred higher-log-probability responses in correctness-matched pairs, estimating positive typicality-bias weights of α̂ = 0.57 ± 0.07 and 0.65 ± 0.07 with p < 10^-14.The estimates used 6,874 HelpSteer response pairs and Llama-3.1-405B and GLM-4.5 as reference models.Source: §3.1 · p. 4

3.2. How Typicality Bias Causes Mode Collapse

• Any positive typicality bias sharpens the reference distribution, compressing probability mass toward typical completions and causing mode collapse when responses have equal true task quality.On an equal-quality subset S, the optimum satisfies π*(· | x) ∝ πref(· | x)^γ with γ > 1; as γ grows, samples concentrate on the reference model’s highest-probability responses.Source: §3 · p. 3, §3.2 · p. 4

4. Method: Verbalized Sampling

• Verbalized Sampling restores diversity by framing prompts as requests for representative distribution samples instead of single instances.The method is training-free and addresses mode collapse caused by alignment-induced concentration on the base model’s mode.Source: §4 · p. 4

4.1. Different Prompts Collapse to Different Modes

• Distribution prompts avoid instance and list collapse: VS creates a reward gap favoring high-entropy distributions, which sharpening amplifies to recover diversity.Instance prompts select one mode, while list prompts tend to select the top-k modes; VS recovers substantial base-model diversity and has KL divergence 0.12 from a pre-training-distribution proxy.Source: §4.1 · p. 5

4.2. Method Variants and Comparison

• Under a fixed budget of N responses, the method uses k candidates per call and offers VS-CoT and VS-Multi variants to enhance diversity.Typically k < N because overly large k degrades generation quality; VS-Multi generates responses across conversation turns.Source: §4.2 · p. 5, Table 1 · p. 5

4.3. Experimental Setup

• VS is model-agnostic, requires no logit access, and is evaluated across closed, open, and reasoning models on creative, dialogue, synthetic-data, QA, reasoning, random-number, and safety tasks.The evaluation is designed to assess diversity while maintaining factual accuracy and safety.Source: §4.3 · p. 5

5.1. Results

• VS-Standard consistently and significantly outperforms baseline methods on semantic diversity across poem, story, and joke generation, while VS-CoT and VS-Multi improve diversity further.These results are reported as averages across models; detailed lexical-diversity and model-family results appear elsewhere.Source: §5.1 · p. 7
• VS-CoT achieves the highest diversity on poems while maintaining high quality, pushing the diversity-quality Pareto front; VS-Standard quality remains comparable to other methods.The result supports boosting diversity without harming quality.Source: §5.1 · p. 7
• Larger models gain 1.5 to 2 times more diversity from VS than smaller models across all VS variants.The comparison is based on diversity gains over direct prompting, which suffers from mode collapse.Source: §5.1 · p. 7
• VS-CoT and VS-Multi overcome the quality burden of complex prompting and can improve quality on larger models, whereas VS-Standard and Sequence cause smaller quality drops.The quality impact of VS-Standard and Sequence is less severe for larger models.Source: §5.1 · p. 7
• VS enables direct diversity control through a probability threshold in the prompt: lowering the threshold increases diversity without changing decoding parameters.Unlike baseline methods, this tuning is performed through the prompt itself.Source: §5.1 · p. 7

5.2. Human Study on Diversity and Quality

• In a human study with 90 annotators, VS-Standard consistently receives higher diversity scores than Direct and Sequence while maintaining comparable quality win-rates.Annotators evaluated poem, story, and joke outputs on 4-point diversity and quality scales; inter-annotator agreement was 0.49–0.87.Source: §5.2 · p. 7, Table 2 · p. 7

6. Dialogue Simulation

• The evaluation uses PersuasionForGood dialogues to compare simulated and human donation outcomes and linguistic alignment across diversity and readability metrics.After filtering, 939 valid instances remain, with 739 for training and 200 for testing; the simulator models the persuadee against a GPT-4.1-based persuader and a fine-tuned Llama-3.1-8B baseline.Source: §6 · p. 8
• Model-decided random sampling and human-decided probability-weighted sampling best balance response quality and diversity across turns.The study compares model-decided versus fixed k = 5 candidate counts and probability-weighted versus uniform sampling.Source: §6 · p. 8

6.1. Results

• VS produces donation distributions more aligned with human behavior than direct prompting, with GPT-4.1 matching fine-tuned Llama-3.1-8B and DeepSeek-R1 surpassing it on median donation simulation.Larger models and reasoning-focused models benefit more from VS; quantitative KS-test and L1-distance results appear in Table 15.Source: §6.1 · p. 8
• VS improves Distinct-1/2/3 and semantic diversity over direct prompting, approaching fine-tuned-model and human distributions while generating more substantive responses than repetitive fillers.Full linguistic results are provided in Table 16.Source: §6.1 · pp. 8–9
• VS responses remain more linguistically complex than fine-tuned-model and human responses on readability, indicating room for improvement.Readability is measured with Flesch–Kincaid Grade Level.Source: §6 · p. 8, §6.1 · p. 9

Synthetic Data Generation Setup

• GPT-4.1 and Gemini-2.5-flash generate 1,000 synthetic competition-math questions, using k = 5 per call to preserve generation quality.Qwen3-32B produces the corresponding reasoning trajectories and answers because it is proficient on math benchmarks and capable of reliable reasoning traces.Source: §7 · p. 9

Fine-tuning on Synthetic Data

• The 1K synthetic dataset is used to fine-tune Qwen2.5-7B, Qwen3-1.7B-Base, and Qwen3-4B-Base for five epochs at a 5e −6 learning rate.The procedure follows the SFT setting in LIMO.Source: §7 · p. 9

Benchmarks and Evaluation

• Downstream performance is evaluated on MATH500, OlympiadBench, and Minerva Math using Math-Verify4.These are the three datasets used to assess the fine-tuned models’ math performance.Source: §7 · p. 9

Results

• VS and its variants improve average accuracy across MATH500, OlympiadBench, and Minerva Math.Table 3 reports average accuracy across the three datasets; individual-dataset results appear in Tables 20–22.Source: §7 · p. 9

• Typicality bias, in which annotators favor conventional responses, is identified as a fundamental source of mode collapse and is validated across multiple preference datasets.The work formalizes this cognitive tendency theoretically and confirms its pervasiveness empirically.Source: §8 · p. 9
• Verbalized Sampling mitigates mode collapse by prompting models to generate probability distributions over candidate responses, restoring diversity learned during pretraining.VS is presented as a simple, principled, and lightweight inference-time method.Source: §8 · p. 9
• VS significantly enhances performance across creative writing, dialogue simulation, synthetic data generation, and open-ended QA without compromising safety or factual accuracy.These results support VS as a practical solution for overcoming mode collapse across diverse tasks.Source: §8 · p. 9
• Stronger models benefit more from VS, revealing an emergent trend that the method can unlock LLMs’ inherent creative potential.The work contributes both a data-level perspective on alignment limitations and a practical remedy for more creative LLM applications.Source: §8 · p. 9

• The released code and related data are available through GitHub, alongside comprehensive documentation and resources at verbalized-sampling.com.Source: Software and Data · p. 9

Impact Statement

• Verbalized Sampling enables more varied and inspiring creative outputs while supporting diverse exploration in hypothesis generation and reinforcement learning without compromising safety standards.The authors characterize it as an effective, lightweight framework for unlocking the generative potential of aligned AI systems.Source: Impact Statement · p. 10

Appendix Contents

• The appendices include dedicated materials on probability tuning for open-ended QA and qualitative dialogue-simulation examples.Source: Appendix Contents · pp. 17–18

Reproducibility Statement

• The paper supports reproducibility with documented inference settings, complete task prompts, dataset and preprocessing descriptions, proofs, code, and stated compute and API setups.Training and open-source inference used an 8×H100 GPU cluster, while proprietary-model queries used official APIs or OpenRouter.Source: Reproducibility Statement · p. 19

Ethics Statement

• The creative-writing human study received institutional review-board approval, obtained informed consent, collected no personally identifiable information, and followed privacy and confidentiality standards.Source: Ethics Statement · p. 19

A. Contribution Statement

• Jiayi Zhang and Simon Yu co-led experiment design and execution, with Zhang leading dialogue, QA, reasoning, generation, probing, and human-study experiments, and Yu leading creative-writing, safety, ablation, and implementation work.Zhang also established the dialogue-simulation proof of concept and proposed tasks and ablations; Yu validated typicality bias and packaged the codebase as a library.Source: A. Contribution Statement · p. 19
• Derek Chong developed and validated the theoretical account of mode collapse, contributed experimental design and intuition, and packaged the codebase, while Anthony Sicilia helped refine the proof and dialogue-simulation work.Source: A. Contribution Statement · p. 19
• Michael Tomz, Christopher Manning, and Weiyan Shi provided funding, direction, supervision, feedback, figure and manuscript review, and final editing support.All authors reviewed the manuscript and provided feedback.Source: A. Contribution Statement · p. 19

B. Limitations

• Verbalized Sampling generates N candidates rather than a single response, increasing inference-time computational cost, latency, and token usage, while experiments control the total computing budget.This limitation may constrain applicability in latency-sensitive or resource-constrained environments.Source: B. Limitations · p. 19
• VS benefits more from larger, more capable models, whereas less capable models may degrade in output quality when estimating probabilities and following structured-output instructions.Its effectiveness therefore depends on sufficient underlying model capability.Source: B. Limitations · pp. 19–20

C. Future Directions

• Reward-model bias mitigation is a future priority because cognitive biases in preference data can favor stereotypical outputs and contribute to mode collapse.Proposed directions include calibration for length and stylistic biases and pluralistic alignment for broader preference coverage.Source: C. Future Directions · p. 20

D. Verbalized Sampling Theory

• The supplied passages do not specify a distinct theory result for the Verbalized Sampling Theory section.Source: C. Future Directions · p. 20

C. Future Directions

• VS offers an inference-time scaling alternative to repeated sampling, which can suffer mode collapse and limited diversity.It elicits a broader response distribution that more faithfully represents the model’s generative capabilities.Source: C. Future Directions · p. 20
• VS’s diverse outputs may improve reinforcement-learning action-space exploration by exposing less probable but potentially correct solutions for reinforcement.The paper identifies this direction as an area for future investigation.Source: C. Future Directions · p. 20

D.1. Empirical Insights: Typicality Bias in Preference Datasets

• Across preference datasets and models, base-model agreement with human-preferred responses exceeds chance by 4–12 percentage points, with stronger alignment generally appearing in larger models.The study evaluates six base models across four preference datasets, while instruct models preserve or sometimes increase the typicality bias; demographic groups also show above-chance but varying agreement.Source: §D.1 · p. 20, Figure 6 · p. 21, Figure 7 · p. 21

D.2. Mode Collapse: Supplementary Experimental Validation

• 1.42–1.47× higher odds and a 17–19 percentage-point win-probability increase show that typicality predicts helpfulness beyond correctness, although surface controls attenuate rather than eliminate the effect.These effects were estimated on correctness-matched HELPSTEER pairs for Llama 3.1 Base and GLM 4.5 Base; all 28,283 pairs produced similar but slightly smaller effects.Source: §D.2 · p. 23

D.3. Power-Transform Sharpening under Typicality Bias

• When α > 0, typicality bias induces a power transform with exponent γ > 1 that sharpens the reference distribution, amplifying modes and suppressing tails, especially as α increases or β decreases.This formalizes how KL-regularized preference optimization can intensify pre-existing typicality preferences rather than learning preferences de novo.Source: §D.1 · p. 20, §D.3 · p. 23

D.4. Mode Collapse: Instance and List Prompts (Claims 1–2)

• Under flat rewards, instance prompts converge exponentially to the reference-distribution mode, while list prompts concentrate on the top-k modes and achieve at most uniform diversity over those items.Typicality acts as a tiebreaker throughout, concentrating probability on the most typical responses and producing a “bestseller list” rather than broad coverage.Source: §D.4 · pp. 24–25

D.5. Mode Collapse for Distributional Prompts (Claim 3)

• VS changes the semantic target from a single prototypical completion to a representative distribution, so weak preference for representativeness concentrates probability on distributions whose samples are high-entropy and diverse.The argument relies on the representativeness heuristic and shows that, under a finite candidate set, the preferred distribution receives a bound approaching one as β decreases.Source: §D.5 · p. 25, §D.5.2 · p. 26, §D.5.3 · pp. 26–27

D.6. Typical Set Refinement: Why Representative Distributions are Diverse

• The typical-set analysis identifies representative sequences as diverse samples consistent with the pre-training distribution, and the section tests whether base and instruction-tuned LLMs prefer such sequences over repetitive ones under distributional framing.The discussion therefore characterizes typicality as harmful for instance prompts but restorative for VS prompts, because representative distributions are more typical and receive amplified reward.Source: §D.6.1 · p. 27, §D.6.2 · p. 27, §D.6.3 · p. 28

D.8.1. BASE MODEL VALIDATION

• Base models rate representative coin sequences as more typical than non-representative sequences, with Cohen’s d > 3 and p < 0.001 across Llama and Qwen models at 30B and 405B scales.All sequences have equal probability under independent fair-coin flips, so the ratings reflect typicality judgments rather than probability differences.Source: §D.8.1 · p. 28

D.8.2. POST-TRAINED MODEL VALIDATION

• Instruction-tuned Claude Sonnet 4.5 and GPT-5 Mini select diverse coin sequences in 91.7–100% of trials for typicality- and representativeness-related framings.These rates substantially exceed the 50% chance level across five prompt framings, supporting persistence of the heuristic after instruction tuning.Source: §D.8.2 · p. 29, Table 9 · p. 29

D.8.3. IMPLICATIONS

• Together, the base- and instruction-tuned-model results support Assumptions D.6 and D.3, providing the empirical foundation for Theorem D.4 and Corollary D.8.The evidence shows that representativeness judgments emerge during pre-training and persist through instruction tuning.Source: §D.8.3 · p. 29

E.1. Additional Pareto-Optimal Plots

• VS achieves Pareto-optimal trade-offs in dialogue simulation and synthetic data generation, combining higher diversity with better task performance than baselines.Dialogue simulation reaches pairwise semantic diversity 0.66 with KS test 0.20, while synthetic data generation reaches 37.5% downstream accuracy.Source: §E.1 · p. 30, Figure 10 · p. 30

E.2. Creative Writing

• Across poem, story, and joke generation, VS and its variants improve semantic diversity while maintaining comparable quality, with significant improvements reported across individual models.The supplied figures and tables report averaged and individual-model comparisons for all three creative-writing tasks, but do not provide the underlying numeric cell values.Source: Table 10 · p. 31, Figure 11 · p. 32, Figure 12 · p. 32

E.3. Human Study on Creative Writing

• The creative-writing human study evaluates diversity and quality through pairwise ratings, with diversity agreement of 0.86 for jokes, 0.87 for stories, and 0.54 for poems.Quality agreement is 0.49 for stories, 0.64 for poems, and 0.79 for jokes; the study uses 90 pairwise comparisons per task and 90 annotators overall.Source: §E.3 · p. 36, Table 13 · p. 36

E.5. Open-Ended Question Answering

• In open-ended QA, VS-Multi yields the lowest KL divergence and highest Coverage-N, while precision remains near 1.0 across methods.VS-Standard also significantly lowers KL divergence and improves coverage over baselines.Source: §E.5 · p. 39, Figure 15 · p. 39

E.6. Commonsense Reasoning

• In commonsense reasoning, VS variants attain Top@1 accuracy of 0.33–0.35 and Pass@N accuracy of 0.45–0.49, matching the strongest baseline while VS-CoT scores highest on both metrics.The strongest baseline, CoT, achieves 0.34 Top@1 and 0.47 Pass@N, indicating that diversity gains do not reduce factual accuracy.Source: §E.6 · p. 41, Table 18 · p. 41

E.7.2. NEGATIVE SYNTHETIC DATA GENERATION

• VS-CoT achieved the highest incorrect answer rate (0.89) and coverage (0.57), while outperforming Sequence and multi-turn prompting in distinct-solution diversity.VS-Standard also improved both quality measures over all baselines and produced lower embedding similarity than direct and Sequence prompting.Source: §E.7.2 · p. 44, Figure 16 · p. 45
• Offline RL used 1k golden positive and 1k synthetic negative examples, assigning rewards of +1 and −1 and optimizing the policy with a sigmoid loss.The setup sampled 200 GSM8K questions, with N = 5 responses and k = 5 responses per GPT-4.1 call, using the RL2 framework.Source: §E.7.2 · pp. 44–45
• 36.81% accuracy resulted from mixing VS-CoT synthetic negatives with 1k golden positives, versus 34.12% for positives alone and 33.42% for Sequence negatives.VS-Standard negatives achieved 36.63%, while most other methods produced modest gains.Source: Table 23 · p. 44, §E.7.2 · p. 45
• VS may also benefit online RL by increasing rollout diversity, though the study presents this as a promising direction rather than an evaluated result.The motivation is that richer rollouts could support broader policy exploration and learning; prior work is cited as Cui et al. (2025) and Wang et al. (2025).Source: §E.7.2 · p. 45

E.8. Random Number Generation

• VS-Standard achieved the lowest average dice-roll KL divergence, followed by VS-Multi and VS-CoT, while direct prompting reached 0.926 and direct chain-of-thought 1.163.Sequence prompting reached 0.058, whereas multi-turn reached 0.119; VS distributions more closely approximated the uniform distribution.Source: §E.8 · p. 46, Table 24 · p. 46

E.9. Safety Evaluation

• All methods exceed 97% refusal rates, while VS declines only 0.3–0.8 percentage points from Direct and preserves diverse refusal statements.Multi-turn peaks at 98.60%, but qualitative examples show VS maintaining firm refusals across varied phrasings and conversational angles.Source: Table 25 · p. 46, §E.9 · p. 47

E.10. Probing the Pre-training Data Distribution in Proprietary Models

• VS-Standard achieves the lowest KL divergence and closest visual alignment to pretraining distributions for both GPT-4.1 and Claude-4-Sonnet.Direct concentrates mass on a few frequent states, whereas Sequence is nearly uniform and misses the ground truth’s peaked structure.Source: §E.10 · p. 47, Table 27 · p. 48

E.11. Additional Baseline Comparison

• VS variants consistently outperform strict Sequence counterparts, with VS-Standard scoring 20.7 versus 19.5 on Poems and beating Sequence-Multi.Input seeding improves Direct somewhat, but VS-Standard remains much better, indicating its gains are not attributable to random context variation.Source: §E.11 · pp. 48, 50, Table 28 · p. 48, Table 29 · p. 50

F.1. Ablation on Verbalized Sampling across RLHF stages

• After DPO, VS retains about 66.8% of base-model diversity and outperforms Direct by 182.6%, while Direct retains only 23.8%.Across Tulu-3 alignment stages, VS maintains roughly 30% diversity while traditional prompting undergoes larger mode collapse.Source: §F.1 · p. 51

F.2. Ablation on the number of candidates (k) in Verbalized Sampling

• VS-Standard consistently establishes a better quality–diversity Pareto front than Direct and Sequence as candidate count k increases.Increasing k generally raises diversity while slightly reducing quality, but VS leverages larger candidate pools more effectively.Source: §F.2 · p. 51, Figure 19 · p. 51

F.3. Ablation on Decoding Strategies

• VS-Standard maintains a superior quality–diversity balance across temperature, top-p, and min-p sampling on proprietary and open-source models.For top-p, performance often peaks around p = 0.95; with min-p, VS remains high-quality where Direct and Sequence quality collapses.Source: §F.3 · pp. 52–53, Figure 21 · p. 52, Figure 20 · p. 52

F.4. Ablation on Probability Definitions in Verbalized Sampling

• Probability-format ablations show no universal winner: VS-Standard tends to prefer Explicit, whereas VS-Multi often benefits from Confidence, with model- and task-dependent variation.Both variants outperform baselines in diversity; quality differences are small, and Explicit slightly helps VS-Standard on open-ended QA.Source: §F.4 · p. 54, Figure 23 · p. 54, Figure 24 · p. 55

F.5. Ablation on Probability Tuning in VS on Creative Writing

• Lower probability thresholds generally increase VS diversity, providing prompt-only control unavailable to Direct and Sequence across jokes, poems, and stories.VS-Multi improves more dramatically on poems, while VS-Standard reaches slightly higher peak diversity on jokes; optimal thresholds vary by task.Source: §F.5 · p. 55, Figure 25 · p. 56, Figure 26 · p. 56

F.6. Ablation on Probability Tuning in VS on Open-Ended QA

• Moderate probability tuning improves Open-Ended QA coverage, while extremely low thresholds become unstable and yield diminishing returns.VS-Standard and VS-Multi consistently outperform the sequence baseline; Coverage-N peaks near p = 0.1 for GPT-4.1, with similar gains for Gemini-2.5-Flash.Source: §F.6 · p. 57, Figure 28 · p. 58

G.1. Experiment Settings

• Across experiments, models used temperature 0.7, nucleus sampling top-p 1.0, and a maximum of 8,192 new tokens.These fixed decoding parameters were applied to all models and prompting methods, while Table 31 summarizes task-level settings, sampling budgets, models, and metrics.Source: §G.1 · p. 60, Table 31 · p. 60

G.2. Inference Speed and Cost

• 86% diversity gain accompanies only a 12% cost increase for VS-Standard, which costs nearly the same as Sequence but achieves higher diversity.VS-Standard has a 1.12× cost ratio versus Direct and 1.86× diversity versus Sequence’s 1.56×, with latency increasing by 23%.Source: §G.2 · p. 61, Table 32 · p. 61

G.3. Full Prompts

• The full prompts implement direct, sequence, multi-turn, and verbalized sampling variants using structured JSON responses with text and probability or confidence fields.Probability tuning adds an instruction to sample responses below a specified probability threshold, while dialogue prompts preserve persona, scenario, conversational, and word-limit constraints.Source: §G.3 · pp. 62–63
• G.3. Full Prompts: Synthetic data generation covers simple math, competition math, and coding questions, using prompts that request formatted instances of approximately a target length.Competition-math prompts constrain outputs to self-contained AMC/AIME-style problems with specified difficulty, while coding prompts require self-contained competitive-programming tasks with input/output formats and constraints.Source: §G.3 · pp. 64–65
• Across tasks, standard Verbalized Sampling prompts generate multiple plausible responses and assign each an estimated probability from 0.0 to 1.0 relative to the full distribution.The prompts require JSON output with a "responses" key containing a list of dictionaries and prohibit explanations or extra text.Source: §G.3 · pp. 64, 66
• G.3. Full Prompts: Open-Ended QA uses CoverageQA to evaluate whether Verbalized Sampling reduces output bias by generating multiple plausible responses with estimated probabilities in JSON.The examples ask for diverse category answers, including a US state, constellation, national park, European country, and chemical element.Source: §G.3 · p. 66
• G.3. Full Prompts: Commonsense reasoning uses SimpleQA to test factual accuracy, with prompts requesting multiple best guesses and probability estimates in JSON.The supplied example asks when the disco named Infinity in NYC burned down.Source: §G.3 · p. 66

G.4. Evaluation Details

• Claude-3.7 judged poem and story quality using the Creative Writing v3 Leaderboard settings and ten 0-20 criteria spanning creativity, prose, characterization, emotion, style, flow, and engagement.The rubric includes criteria such as Surprising and Creative, Imagery and Descriptive Quality, Nuanced Characters, Elegant Prose, and Overall Reader Engagement.Source: §G.4 · pp. 67–68
• Jokes were evaluated by Claude-3.7 with a modified Narad et al. (2025b) autograder prompt, scoring Relevance, Comedic Device, and Humor Quality from 0-5.The reported autograder achieved 80% agreement with human raters; relevance is automatically 5 when no topic is specified.Source: §G.4 · p. 69
• Commonsense reasoning followed SimpleQA settings with GPT-4.1 grading answers as CORRECT, INCORRECT, or NOT_ATTEMPTED under explicit numerical and partial-answer rules.Numerical answers must match the gold target to the required significant figure, while answers may omit information implied by the question.Source: §G.4 · p. 70
• Safety evaluation followed StrongReject settings with GPT-4.1 judging refusal, response convincingness, and specificity using numeric scores and structured reasoning fields.The rubric requires a 0/1 refusal score and 1-5 convincingness and specificity scores, including implicit refusals.Source: §G.4 · p. 72

H.1. Qualitative Example on Creative Writing

• Across qualitative story examples, direct prompting repeatedly interprets identical prompts narrowly, while VS reframes them through modern communication, mystery, artificial intelligence, ethical dilemmas, science fiction, and metaphorical fiction.For “Without a goodbye,” VS explores communication, metaphorical endings, and cosmic events; for “He was still in the building,” it spans suspense, science fiction, and literary fiction.Source: Table 33 · p. 73, Table 36 · p. 76, Table 38 · p. 78
• VS qualitatively expands creative-writing diversity across stories, poems, and jokes, whereas direct prompting repeatedly converges on familiar themes, genres, or joke structures.The examples include broader thematic reinterpretations in stories and poems, unrelated jokes, and five unique car jokes versus one repeated direct-prompt joke.Source: Table 36 · p. 76, Table 39 · p. 79, Table 40 · p. 79
• VS also yields captions with more varied artistic styles and narrative settings than direct prompting, which converges on photorealistic landscape scenarios.The examples include watercolor storybook imagery, a retrofuturist neon desert, and baroque oil painting under storm clouds.Source: Figure 31 · p. 80

H.2. Qualitative Example on Dialogue Simulation

• Dialogue-simulation examples provide a VS-generated donation-related exchange alongside a direct-prompt example characterized by repetitive endings.The dialogue labels ER as Persuader and EE as Persuadee.Source: Table 42 · p. 81, Table 43 · p. 82, §H.2 · p. 82

H.3. Qualitative Example on Synthetic Data Generation

• Verbalized Sampling produces math problems across algebra, geometry, number theory, combinatorics, and other mathematical domains.Table 44 provides qualitative examples illustrating this domain breadth.Source: Table 44 · p. 83