Source-linked AI summary
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias
Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander Ratner, Ranjay Krishna, Jiaming Shen, Chao Zhang
TL;DR
Simple class-conditional prompting can limit diversity and reproduce systematic biases in LLM-generated training data. The paper introduces AttrPrompt, which interactively constructs and randomly combines data attributes for high-cardinality classification datasets. AttrPrompt outperforms SimPrompt across evaluated tasks and matches its performance with only 5% of the querying cost, while attribute diversity improves downstream performance.
Problem
Simple class-conditional prompts may limit generated-data diversity and inherit systematic LLM biases, leaving attributed training-data generation underexplored.
Method
AttrPrompt uses interactive LLM-assisted attribute design, ambiguity filtering, and random combinations of attributes to generate classification training data.
Results
AttrPrompt significantly outperforms SimPrompt across four classification tasks and achieves SimPrompt’s performance with only 5% of its ChatGPT querying cost.
Takeaways & Limitations
Attribute diversity is important for downstream performance, while attributed prompts can improve generated-data quality and budget efficiency.
Takeaways & Limitations
The findings focus primarily on text classification, so generalizability to other data types and tasks remains unexplored.
Abstract
from arXiv · showhide
Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. While previous research has explored different approaches to training models using generated data, they generally rely on simple class-conditional prompts, which may limit the diversity of the generated data and inherit systematic biases of LLM. Thus, we investigate training data generation with diversely attributed prompts (e.g., specifying attributes like length and style), which have the potential to yield diverse and attributed generated data. Our investigation focuses on datasets with high cardinality and diverse domains, wherein we demonstrate that attributed prompts outperform simple class-conditional prompts in terms of the resulting model's performance. Additionally, we present a comprehensive empirical study on data generation encompassing vital aspects like bias, diversity, and efficiency, and highlight three key observations: firstly, synthetic datasets generated by simple prompts exhibit significant biases, such as regional bias; secondly, attribute diversity plays a pivotal role in enhancing model performance; lastly, attributed prompts achieve the performance of simple class-conditional prompts while utilizing only 5\% of the querying cost of ChatGPT associated with the latter. The data and code are available on \url{https://github.com/yueyu1030/AttrPrompt}.
1 Introduction
Prior LLM-generated training data typically relies on simple class-conditional prompts, which may limit diversity and inherit systematic biases. This paper proposes diversely attributed prompts and finds stronger performance, greater diversity, and substantially lower querying cost.
- Simple class-conditional prompts may limit generated-data diversity and inherit systematic LLM biases.
- 68.01% of SimPrompt-generated NYT examples were predicted as North America, versus 0.69% for Africa.The passage describes Africa as 100 times less prevalent than North America.
- Models trained with fixed attributes substantially underperform models trained with randomly varied attributes, highlighting attribute diversity’s importance.
- AttrPrompt identifies attribute dimensions and values, then randomly combines them into prompts replacing SimPrompt.Example attributes include location and style.
- AttrPrompt significantly outperforms SimPrompt across four classification tasks in both generated-only and merged real-plus-generated training scenarios.
- AttrPrompt matches SimPrompt’s performance using only 5% of SimPrompt’s ChatGPT querying cost.
2 Related Work
Related work has used LLMs to generate several data types and has often relied on simple prompts or explicit attributes. AttrPrompt instead targets attributed training-data generation for high-cardinality topic classification with semi-automated attribute proposal.
- Prior LLM data-generation applications include tabular data, relation triplets, sentence pairs, and instruction data.
- Existing training-data-generation methods commonly use simple class-conditional prompts while addressing low-quality data through reweighting or selection.
- Attribute-aware text-generation methods generally target tasks such as style transfer and require explicit attribute provision.
- AttrPrompt differs by using a semi-automated strategy that lets LLMs propose attribute values autonomously.
- Discrete prompt-optimization methods mainly target inference, whereas AttrPrompt optimizes prompts for attributed training-data generation.
3 Large Language Model as Attributed Training Data Generator
AttrPrompt generates training data with diverse attribute configurations for high-cardinality, imbalanced datasets spanning multiple domains. Its workflow combines interactive attribute design, ambiguity filtering, randomized prompt construction, and comparison against SimPrompt and Gold data.
- 3.1 Datasets: The study covers topic-classification datasets with 23 to 504 classes across diverse domains.The datasets include NYT, Amazon, Reddit, and StackExchange.
- 3.1 Datasets: The datasets have high imbalance ratios, reflecting long-tail class distributions in real applications.
- 3.2 Interactive Attribute Generation: ChatGPT interactively proposes attribute dimensions, after which humans select attributes and values for data generation.Example dimensions include subtopics, length, location, reader group, style, and time.
- 3.2 Interactive Attribute Generation: Class-Dependent Attribute Value Filtering removes values related to similar classes to reduce ambiguity.The process queries ChatGPT for the top-5 similar classes and checks each candidate value against them.
- 3.2 Interactive Attribute Generation: Random attribute configurations are inserted into a prompt template and sent to ChatGPT to collect attributed samples.Configurations can combine subtopic, length, style, and location values.
- Experiments compare AttrPrompt, SimPrompt, and Gold with equal generated-data counts, using BERT-base-uncased unless otherwise specified.
- AttrPrompt produces more diverse samples than SimPrompt, which tends to focus news on the United States while AttrPrompt covers locations worldwide.
4 Diversity Analysis of the Generated Data
AttrPrompt generates more diverse data than SimPrompt across lexical and embedding-based measures, although both remain less lexically diverse than the Gold dataset. Experiments also examine how attribute configurations affect downstream performance.
- AttrPrompt has higher vocabulary size than SimPrompt for both the whole dataset and the class average, but both trail Gold.Vocabulary size is used as a measure of lexical diversity.
- The study evaluates diversity using vocabulary size, same-class cosine similarity, APS, and INGF across generated and Gold datasets.For APS, lower values indicate better diversity; for INGF, higher values indicate better diversity.
- Gold has the lowest same-class cosine similarity, while AttrPrompt is lower than SimPrompt and closer to Gold.Lower similarity indicates greater diversity in this comparison.
- Fixing one attribute while randomizing the others is used to study how attribute diversity influences downstream model performance.The experiments generate 50 data points per class under each one-fixed-others-random configuration.
5 Bias Analysis of the Generated Data
The bias analysis finds substantial regional skew in datasets generated with simple prompts, while attributed prompts produce more balanced location distributions. Class-specific predictions also vary noticeably, underscoring the unpredictability of generated-data bias.
- The analysis treats NYT location as a case study and notes that other dataset bias studies appear in Appendix D.2.
- Manual annotations show Gold and SimPrompt are largely biased toward North America, whereas AttrPrompt produces a relatively balanced location distribution.The comparison uses 100 manually annotated samples from each dataset.
- Predicted whole-dataset location distributions resemble the manual annotations, while AttrPrompt remains balanced across specific classes.Gold and SimPrompt show class-specific regional biases, including opposite Oceania–North America trends for tennis.
6 Experiments on the Trained Models
Across classification experiments, AttrPrompt consistently improves over SimPrompt, remains effective when combined with real training data and prior generators, and offers strong budget and sample efficiency. Results also examine robustness across generator and classifier sizes and extend evaluation to multi-label classification.
- 6.1 Training with generated data: AttrPrompt consistently outperforms SimPrompt by 6–10 points, surpasses zero-shot ChatGPT on most accuracy and all F1 comparisons, but generally trails the original training data.The AttrPrompt gain over SimPrompt is statistically significant; generated data remains below the original training set in most cases.
- 6.2 Augmenting existing dataset with generated data: AttrPrompt improves performance in every merged-training case, showing that generated data can complement the original training set.The study combines generated and original data through a simple dataset merge.
- 6.3 The budget and sample efficiency of the generated data: 5% of SimPrompt’s full budget is sufficient for AttrPrompt to match or outperform SimPrompt across all datasets.AttrPrompt also has better sample efficiency than SimPrompt, although Gold data performs better in the high-data regime.
- 6.4 The performance with respect to model parameter size: AttrPrompt outperforms SimPrompt across generator and classifier sizes, while larger generators generally improve performance and a 66M-parameter classifier often suffices.GPT-4 results use only 10% of the full generated dataset and are therefore not directly comparable with the other generator settings.
- 6.5 Plugging AttrPrompt in existing approaches: AttrPrompt consistently boosts existing generation methods by more than 5% on fine-grained tasks and improves ReGen by 14%–26% absolute.The gains extend to approaches that do not use LLMs, supporting AttrPrompt as a plug-in module.
- 6.6 Multi-label classification: The study also evaluates SimPrompt and AttrPrompt on a 98-class arXiv multi-label classification dataset using Micro/Macro-F1, Precision@k, nDCG@k, and MRR.The corresponding results are reported in Table 10.
7 Limitation, Social Impact, and Future Work
The paper discusses limitations, social implications, and future directions for attributed prompts. Its scope is mainly text classification, while broader modalities and more automated attribute discovery remain open areas.
- Limitations: The study primarily evaluates attributed prompts for text classification, so generalizability to other data types and tasks remains unexplored.
- Limitations: AttrPrompt assumes access to target-task label names and requires some human effort to create data attributes.
- Limitations: LLM-generated text can hallucinate plausible but inaccurate information, potentially propagating misinformation in sensitive applications such as news generation.
- Future Work: Future work could test attributed prompts on image, audio, and other classification modalities.
- Future Work: Future work could automate attribute-dimension and attribute-value discovery and selection for improved performance.
8 Conclusion
The paper studies complex attributed prompts for generating training data in high-cardinality, diverse-domain, and class-imbalanced settings. It reports superior performance over simple class-conditional prompts and examines bias, diversity, and efficiency.
- 8 Conclusion: The study investigates complex attributed prompts designed to produce diverse, attributed generated data.
- 8 Conclusion: Attributed prompts outperform simple class-conditional prompts on datasets with diverse domains, high cardinality, and class imbalance.
- 8 Conclusion: The empirical study evaluates training-data generation through bias, diversity, and efficiency.
A Dataset Preprocessing
The preprocessing constructs topic-classification datasets from Reddit, StackOverflow, and arXiv sources. It defines target classes, splits available examples, and specifies SimPrompt and AttrPrompt generation procedures for multi-label data.
- Dataset Selection: Reddit and StackOverflow target topics retain classes with more than 65 original examples, using 50 test examples and limited validation examples per class.
- Dataset Selection: The arXiv dataset covers five subject categories and contains 98 concrete classes after removing generic topics.
- SimPrompt: SimPrompt samples the label count from the original distribution and randomly selects that many classes for each generated document.
- AttrPrompt: AttrPrompt conditions generation on subtopics, technique, writing style, and length for multi-label documents.
- AttrPrompt: Subtopics with fuzzy-matching similarity above γ = 90 are merged, allowing some subtopics to correspond to multiple classes.
B Implementation Details
The implementation uses specified hardware, software, generation settings, and classifier-training procedures. Additional experiments examine encoders, temperature, model size, and long-tail-class performance, while noting a remaining class-balance question.
- Implementation: Experiments run with an Intel i7-5930K CPU, NVIDIA RTX A5000 GPUs, Python 3.8, Huggingface 4.6.0, and PyTorch 1.10.
- Implementation: ChatGPT generation uses top_p = 1.0 and temperature t = 1.0, while classifier training uses AdamW, 5% warmup, decay, and 128-token sequences.
- Implementation: Synthetic-data models train for six epochs using the final checkpoint, whereas clean-data models use the best validation checkpoint after six epochs.
- Additional Analyses: Linear probing experiments compare four pretrained encoders, with AttrPrompt consistently outperforming SimPrompt and surpassing Gold on NYT.
- Additional Analyses: Temperature experiments compare generation diversity from temperature changes against the effects of AttrPrompt on NYT.
- Additional Analyses: Figure 10 plots per-class F1 for Gold, SimPrompt, and AttrPrompt across NYT classes ordered by training-example counts.
- Additional Analyses: Further study of generating or augmenting class-balanced training sets for long-tail performance is left for future work.
C.4 Experiment results on other datasets
Across four low-cardinality classification tasks, AttrPrompt achieves competitive downstream performance and outperforms SimPrompt while using standard training objectives.
- The evaluation generates 6000 examples evenly distributed among classes for classifier fine-tuning.DistilBERT is fine-tuned for five epochs with a learning rate of 2 × 10−5.
- AttrPrompt demonstrates competitive performance across SST-2, IMDB, Yelp, and AG News.The first three datasets are binary sentiment tasks, while AG News has four classes.
- AttrPrompt remains promising against noisy-aware baselines despite using standard cross-entropy loss.The authors identify prompt diversity as an alternative to dedicated noisy-robust learning objectives.
D.1 Regional Bias with Manual Annotations
Manual annotation validates the location-attribute analysis and shows that AttrPrompt better follows specified locations than SimPrompt, whose generated data contain more unknown locations.
- Greater than 0.95 accuracy is obtained for manual-versus-classifier location agreement on Gold and AttrPrompt data, compared with 0.82 for SimPrompt.The lower SimPrompt agreement is attributed to more generated samples with unknown locations.
- Manual location annotations cover 100 samples from each of the Gold, SimPrompt, and AttrPrompt datasets.Unknown is included as an annotation option for text without clear location specifications.
- 0.96 is ChatGPT’s accuracy in following the location attribute specified by AttrPrompt prompts.This accuracy is computed by comparing prompted locations with human annotations.
- Gold and SimPrompt datasets show different levels of attribute bias, whereas AttrPrompt has relatively balanced attribute distributions.Figure 11 summarizes predicted attribute distributions across datasets.
E The Prompt Format for LLM-Generated Attributes
The prompt format combines class information with diverse attributes selected for each domain, using LLM-assisted attribute discovery and varied prompt instantiations to broaden generated data.
- E.1.1 Key Attributes: Key news attributes include subtopics, writing style, sources, geographical locations, timeliness, multimedia, and length.These dimensions are presented as ways to vary generated NYT news beyond class labels.
- E.1.3 Writing Styles: NYT writing styles include investigative reporting, opinion/editorial, human interest/profiles, long-form narrative, and breaking news.
- E.2.1 Key Attributes: Amazon review attributes include product type, brand, user experience, quality and durability, and features and functionality.The selected Amazon attributes are product name, brands, user experience, writing styles, and length.
- E.2.2 Selected Attributes: Amazon review prompts vary product names or brands, usage experiences, and writing styles.The appendix provides prompt queries and corresponding experience and writing-style categories.
- E.2.6 Prompt Template: The appendix specifies separate prompt templates for Amazon Product Reviews and Reddit data.These templates are referenced in Tables 17 and 18.
- E.3.1 Key Attributes: Reddit attributes include informative guides, writing style, discussion starters, personal experiences, and resources or recommendations.Examples span educational, entertaining, discussion-oriented, and personal-story formats.
- E.3.4 Writing Styles: Reddit writing styles include informative/educational, entertaining/funny, thought-provoking/discussion, and personal/storytelling.
G.3.2 Experience
The experience attributes enumerate concrete scenarios for Reddit and StackExchange categories, covering hardware building, gaming, networking, multiplayer systems, and procedural generation.
- Reddit experience attributes include DIY PC builds, budget-friendly builds, cable management, RGB lighting, troubleshooting, quiet PCs, workstations, and water-cooling.
- Additional PC-building scenarios cover custom builds, component selection, case modding, compact systems, home servers, multimonitor setups, and gaming peripherals.
- Xbox experience attributes include exclusive games, media applications, memorable gaming moments, online purchases, Kinect, and Xbox Play Anywhere.
- Metal Gear Solid scenarios include weapon customization, character development, stealth takedowns, tactical planning, memorable characters, and stealthy gadgets.
- Other Metal Gear Solid attributes cover emotional storytelling, espionage tactics, lore and mythology, and groundbreaking game design.
- StackExchange scenarios include multiplayer cheating, player behavior, matchmaking, lag, connectivity, communication, and moderation issues.
- Procedural-generation scenarios address algorithm efficiency, memory use, randomized content, machine learning, 3D models, game worlds, NPCs, and terrain.
- Additional procedural-generation scenarios concern user-generated content, multithreading, and compatibility across game engines.