Scaling Synthetic Data Creation with 1,000,000,000 Personas

• At scale, quantity can grow more easily than diversity, which requires many distinct prompts; prior prompt-diversification paradigms have practical limits.Sampling is described as orthogonal to the work, and sampling alone usually introduces limited diversity.Source: §1 · p. 2
• Seed-based methods struggle to extend beyond limited seed corpora, while key-point methods require impractical exhaustive curation except in narrow domains.The two paradigms diversify prompts using seed instances or curated lists of key points, respectively.Source: §1 · p. 2
• Adding a persona can steer an LLM toward a corresponding perspective, enabling distinctive synthetic data and offering a route to all-encompassing synthesis with a comprehensive persona collection.Persona Hub is also framed as a compressed form of world knowledge distributed across its personas.Source: §1 · p. 2
• Persona Hub comprises 1 billion diverse personas automatically constructed from web data—about 13% of the world's population—that can access LLM perspectives and combine with almost any synthesis prompt.The paper presents personas as distributed carriers of world knowledge, each associated with distinctive knowledge, experiences, interests, personality, and profession; this approach is not limited by seed-corpus size.Source: §1 · p. 3
• The paper showcases persona-driven synthesis of math and logical reasoning problems, instructions, knowledge-rich texts, game NPCs, and tools, presenting it as versatile, scalable, flexible, and easy to use.The authors state that the methodology applies to almost any popular LLM.Source: §1 · p. 3

2.1 Text-to-Persona

• Text-to-Persona prompts an LLM to infer who might read, write, or otherwise engage with a web text, yielding diverse personas at scale.Input-text detail and prompt settings can shape persona granularity; the method can yield billions or even trillions of personas from massive web text.Source: §2.1 · pp. 3–4, Figure 3 · p. 4, Figure 4 · p. 4

2.2 Persona-to-Persona

• Persona-to-Persona supplements web-visible personas by deriving less-visible personas through interpersonal relationships, then expanding each seed persona through six iterations.Examples include deriving a child from a nurse and a movie crew member from a lead actor.Source: §2.2 · pp. 4–5, Figure 5 · p. 5

2.3 Deduplication

• The pipeline applies MinHash deduplication on unigram features at a 0.9 threshold, followed by embedding-based filtering of pairs with cosine similarity greater than 0.9.The MinHash signature size is 128; the similarity threshold can be tightened when fewer instances but greater diversity are needed.Source: §2.3 · p. 5
• 1,015,863,523 personas remained after deduplication and heuristic filtering of low-quality descriptions, forming Persona Hub.Source: §2.3 · p. 5

3 Persona-driven Synthetic Data Creation

• Persona-driven synthesis inserts a persona into a suitable prompt position to steer the LLM’s perspective and create diverse synthetic data at billion scale.The approach is described as straightforward and compatible with the 1 billion personas in Persona Hub.Source: §3 · p. 5
• Zero-shot prompting uses no demonstrations to leave room for model creativity, while few-shot prompting uses demonstrations to better meet requirements.Source: §3 · p. 6
• Persona-enhanced few-shot prompting more effectively strengthens persona-driven synthesis, but requires deriving a corresponding persona for each demonstration beforehand.Source: §3 · p. 6

4 Use Cases

• Persona Hub use cases include large-scale synthesis of math and logical reasoning problems, instructions, knowledge-rich texts, game NPCs, and tools; the paper details math synthesis and treats other scenarios as prompt adaptations.The authors characterize the persona-driven approach as general and versatile, adaptable to different synthesis scenarios by adjusting the prompt.Source: §4 · p. 6

4.1.1 Demonstrations

• A linguist persona steers a generated problem toward computational linguistics, while prompts can still specify its focus or difficulty.The examples include prompts specifying geometry or Olympiad-level difficulty.Source: §4.1.1 · pp. 6–7, Figure 7 · p. 7
• Personas of math professionals surface advanced, fine-grained mathematical concepts, enabling problems that cover those concepts and are more challenging.The paper notes that large numbers of such personas can be harvested by applying Text-to-Persona to public web texts, particularly math texts.Source: §4.1.1 · p. 7, Figure 8 · p. 7

4.1.2 Evaluation

• Using 1.09 million personas, GPT-4 zero-shot synthesis produced 1.09M math problems, with 1.07M used for training and 11.6K verified synthetic test instances.The held-out 20K problems were filtered for agreement among at least two generated solutions; MATH was not used for synthesis or training, and performance generally followed the scaling-law trend as synthetic training data increased.Source: §4.1.2 · pp. 8–9, Figure 9 · p. 9
• Nearly 80% accuracy on the synthetic in-distribution test set surpassed all tested open-source LLMs, but the authors caution that the results are reference-only.The 11.6K test answers were not absolutely reliable, and the fine-tuned model might be the only evaluated model using in-distribution training data.Source: Table 1 · p. 8, §4.1.2 · pp. 8–9
• 64.9% accuracy on MATH: the synthetic-data-fine-tuned 7B model matches gpt-4-turbo-preview and is outperformed only by five named models.The models named as outperforming it are gpt-4o, gpt-4-turbo-2024-04-09, Claude 3.5 Sonnet, Gemini Pro 1.5 (May 2024), and DeepSeek-Coder-V2-Instruct.Source: Table 2 · p. 9, §4.1.2 · p. 9
• Expert review found 193 of 200 sampled challenging problems valid, for a 96.5% validity rate.Two math experts assessed the sample, which involved high-school and university-level math knowledge points in China.Source: §4.1.2 · p. 9
• Synthesized-problem similarity tracks but remains below persona similarity; more specific prompt constraints raise problem similarity, while two tested LLMs yielded similar results at persona similarity 0.9.For gpt-4o and gpt-35-turbo with highly similar personas, most synthesized problem similarities fell between 0.6 and 0.75, below the personas’ 0.9 similarity.Source: §4.1.2 · p. 10, Figure 10 · p. 10

• Clearly specified requirements let varied personas steer the LLM to generate diverse, requirement-matching, persona-relevant problems, including whimsical Ruozhiba-style examples.The Figure 12 caption notes that its third prompt is simplified; practical prompts must specifically define the Ruozhiba style to obtain the desired data.Source: §4.2 · p. 10, Figure 12 · p. 11
• The persona-driven methodology synthesizes typical logical reasoning problems, as illustrated in Figure 11.Source: §4.2 · p. 10, Figure 11 · p. 11

4.3 Instructions

• Persona Hub simulates diverse human users to generate varied instructions, or prompts, reflecting typical requests for LLM assistance.Source: §4.3 · p. 11
• Zero-shot prompting generates instructions from personas without an existing instruction dataset, while persona-enhanced few-shot prompting uses demonstrations and inferred personas to better resemble real-user instructions.The experiments use WildChat (Zhao et al., 2024) as the source of few-shot demonstrations; 50,000 synthesized instructions from zero-shot and persona-enhanced 2-shot prompting were released.Source: §4.3 · pp. 11–12
• Synthesized first-turn instructions can be extended into simulated user–LLM conversations to support instruction-following and conversational abilities.Source: §4.3 · p. 12
• Selecting two Persona Hub personas for LLM role-play can simulate conversations between two people (Jandaghi et al., 2023).Source: §4.3 · p. 12

4.4 Knowledge-rich Texts

• Persona-driven prompting elicits an LLM’s knowledge and perspective to produce informative, knowledge-rich plain text.Figure 14 illustrates examples of this synthesis approach.Source: §4.4 · p. 12, Figure 14 · p. 13
• Scaling the process with Persona Hub’s 1 billion personas can produce texts covering almost any topic at varied levels of granularity.The resulting texts can benefit LLM pre-training and post-training.Source: §4.4 · p. 12
• Beyond Quora articles, personas can prompt the synthesis of educational reading material they may be interested in.This offers another way to obtain large amounts of knowledge-rich text.Source: §4.4 · p. 12

4.5 Game NPCs

• Game designers can project Persona Hub’s typically real-world personas into a game’s characters when they provide its background and world-building information.This offers a practical way to create diverse NPCs at scale.Source: §4.5 · p. 13
• Using Persona Hub to create NPCs can significantly reduce the effort required for brainstorming during game design.Source: §4.5 · p. 13
• Examples show Persona Hub personas used to create NPCs for World of Warcraft and Moonlight Blade.Source: Figure 15 · p. 13, §4.5 · p. 14, Figure 16 · p. 14

4.6 Tool (Function) Development

• Persona Hub can simulate users to predict needed tools (Cai et al., 2023; Schick et al., 2024), enabling functions to be prebuilt for later requests.When a real user makes a similar request, the LLM can call a pre-built tool instead of building one from scratch.Source: §4.6 · p. 14
• The authors describe persona-driven tool creation as a new solution for better serving users, with potential as LLMs become more democratized and multifunctional.Source: §4.6 · p. 15
• Persona-created tools provide functionalities users may need but LLMs cannot access, expanding the services LLMs can provide.Figure 17 illustrates examples, such as a cab driver needing to check traffic conditions.Source: Figure 17 · p. 14, §4.6 · p. 15
• Tool interface definitions can be converted into code implementations with an LLM, producing pre-built tools for future direct use.The paper illustrates this with Figure 18, describing how the resulting tools avoid rebuilding them from scratch each time.Source: §4.6 · p. 15, Figure 18 · p. 15

5.1.1 Paradigm Shift in Data Creation by Humans and LLMs

• Persona Hub shifts LLMs beyond processing existing data toward creating diverse new data from many perspectives.The authors acknowledge that current LLMs may not match humans in data quality or breadth, but argue that Persona Hub’s diversity and coverage may mitigate synthetic-data harms (Shumailov et al., 2023; Dohmatob et al., 2024) and reduce the data bottleneck.Source: §5.1.1 · pp. 15–16

5.1.2 Reality Simulation

• Persona Hub’s billion personas could simulate real-user needs for product, policy, and profiling scenarios, and support virtual societies for testing policies and social dynamics before implementation.The authors say user-behavior simulation could alleviate cold-start challenges.Source: §5.1.2 · p. 16

5.1.3 Full Memory Access of LLMs

• The billion personas can generate queries and responses that turn an LLM’s parameterized world knowledge into textual synthetic data, potentially decompressing its learned information.The authors caution that current Persona Hub and LLM capabilities are preliminary: hallucinations limit the breadth and quality of generated data, though improvements could enable nearly lossless extraction.Source: §5.1.3 · pp. 16–17

5.2.1 Training Data Security and Threats to Current LLM Dominance

• Near-comprehensive prompted extraction could expose seen training data and replicate leading LLMs’ knowledge and capabilities, potentially shifting competition from data advantage toward advanced technology.The authors describe the extraction as generally lossy.Source: §5.2.1 · p. 17

5.2.2 Miscellaneous

• Persona Hub’s diverse writing styles may make synthetic misinformation harder to distinguish from human writing, worsening contamination risks for research and public information (Pan et al., 2023; Chakraborty et al., 2023).Source: §5.2.2 · p. 17

6 Conclusion and Future Work

• Persona-driven synthesis and Persona Hub, a web-curated collection of 1 billion diverse personas, demonstrate potential to scale synthetic-data creation across scenarios.The authors describe the methodology as a general data synthesis engine for research and practice.Source: §6 · p. 17
• Future Persona Hub versions will add fine-grained persona details, making personas more unique and enabling further synthesis opportunities and personalized conversations.The first version focuses on major aspects; the authors aim for descriptions as detailed as Wikipedia biographies.Source: §6 · p. 17
• Future work will explore multimodal synthesis and whether super personas can guide LLMs beyond existing knowledge.The paper explores only text-based LLMs and presents super-persona guidance as a possible way to tap into LLM super intelligence.Source: §6 · p. 18