PaperlayerView exact revision on arXiv

Source-linked AI summary

Does Writing with Language Models Reduce Content Diversity?

Vishakh Padmakumar, He He

arXiv:2309.05196v3cs.CLcs.CYcs.HCcs.LG

TL;DR

The paper investigates whether shared LLM assistance reduces diversity in collaborative writing, a concern for varied expression and perspectives. Using a controlled experiment comparing Solo, GPT3, and InstructGPT argumentative essays, it finds that InstructGPT—but not GPT3—makes essays more homogeneous and less lexically and substantively diverse. The reduction is attributed mainly to less diverse InstructGPT text, while user-contributed text remains largely consistent.

  • Problem

    The paper asks whether using LLMs for writing reduces the diversity of content produced by different users, potentially limiting perspectives in writing.

  • Method

    A controlled experiment compares argumentative essays written without model help, with GPT3, and with feedback-tuned InstructGPT, using homogenization and diversity metrics.

  • Results

    InstructGPT produces more similar essays with lower lexical and key-point diversity than Solo and GPT3, while GPT3 does not significantly differ from Solo.

  • Takeaways & Limitations

    The findings identify reduced content diversity as an additional axis for evaluating LLMs in interactive writing settings, especially after human-feedback tuning.

  • Takeaways & Limitations

    The study examines single interactions between users and models, so dynamics may change through repeated interactions over time.

Abstract

from arXiv · show

Large language models (LLMs) have led to a surge in collaborative writing with model assistance. As different users incorporate suggestions from the same model, there is a risk of decreased diversity in the produced content, potentially limiting diverse perspectives in public discourse. In this work, we measure the impact of co-writing on diversity via a controlled experiment, where users write argumentative essays in three setups -- using a base LLM (GPT3), a feedback-tuned LLM (InstructGPT), and writing without model help. We develop a set of diversity metrics and find that writing with InstructGPT (but not the GPT3) results in a statistically significant reduction in diversity. Specifically, it increases the similarity between the writings of different authors and reduces the overall lexical and content diversity. We additionally find that this effect is mainly attributable to InstructGPT contributing less diverse text to co-written essays. In contrast, the user-contributed text remains unaffected by model collaboration. This suggests that the recent improvement in generation quality from adapting models to human feedback might come at the cost of more homogeneous and less diverse content.

1 INTRODUCTION

The paper asks whether LLM-assisted writing reduces content diversity and tests this question in a controlled comparison of Solo, GPT3, and InstructGPT writing. InstructGPT increases homogenization and reduces lexical and content diversity, while GPT3 does not significantly differ from Solo.

  • Motivation: LLM-assisted writing may shift content toward less colorful, more standardized expression, motivating a study of diversity in co-written text.The paper frames this concern as an algorithmic monoculture arising when many users rely on the same model.
  • Research design: The experiment compares argumentative essays written without model help, with GPT3, and with feedback-tuned InstructGPT.Participants write essays on topics from the New York Times student opinion series.
  • Findings: InstructGPT essays have lower lexical diversity and reduced diversity in expressed key points than essays from the other two groups.The reduction includes increased repetition of higher-order n-grams and fewer unique key points.
  • Findings: GPT3 produces no statistically significant diversity effect despite similar model contribution, whereas InstructGPT provides less diverse suggestions and text.The result links the observed difference between models to the diversity of their contributed text.
  • Implications: Reduced diversity may harm personal expression and creativity and could create feedback loops when future models train on homogenized content.The authors release essays and recorded model suggestions to support further research.

2 DATA COLLECTION

The study uses controlled argumentative-essay experiments to compare writing alone with assistance from base and feedback-tuned language models. Participants complete randomized tasks across all three settings on open-ended, accessible topics.

  • 2 DATA COLLECTION: The study controls the writers and essay topics, then compares essays written with and without model assistance.This controlled-experiment design is used to study effects on content diversity.
  • 2 DATA COLLECTION: Participants write approximately 300-word argumentative essays on topics from the New York Times Student Opinion series.The topics are open-ended enough to allow diverse responses while remaining accessible to users from varied backgrounds.
  • 2 DATA COLLECTION: The CoAuthor interface presents five model continuations after each TAB request, allowing users to accept and edit one or reject all suggestions.Participants are asked to request suggestions at least five times per essay.
  • 2 DATA COLLECTION: The three settings are Solo, GPT3, and InstructGPT, using a base model and a human-feedback-finetuned model respectively.The models are distinguished because human-feedback finetuning may decrease output entropy and text diversity.
  • 2 DATA COLLECTION: Model continuations are sampled with temperature 0.9 and frequency penalty 0.5 using a higher-entropy decoding strategy.The settings follow the “high randomness” configuration from the CoAuthor study.
  • 2 DATA COLLECTION: Each participant completes one essay in each setting, with randomized setting order and topic assignment.Each essay takes 15 minutes on average; the study collects 10 essays for each of 10 topics per setting, totaling 300 essays.

3 HOW MUCH DO USERS ENGAGE WITH THE MODEL?

Users actively engage with both models, accepting many suggestions and retaining substantial model-generated text in their essays. The study also estimates model contributions to key ideas by summarizing essays and aligning key points to model- or user-written sentences.

  • 3 HOW MUCH DO USERS ENGAGE WITH THE MODEL?: Both GPT3 and InstructGPT are helpful to users, with high suggestion acceptance and a sizeable model-written fraction of essays.Table 1 reports averaged per-essay usage statistics and finds no statistically significant difference in usage between the models.
  • 3 HOW MUCH DO USERS ENGAGE WITH THE MODEL?: Users query the model around 9 times per essay, accept about 70% of suggestions, and receive roughly 35% of essay characters from the model.Participants are asked to query the model at least five times, but acceptance remains optional.
  • 3 HOW MUCH DO USERS ENGAGE WITH THE MODEL?: An independent-samples t-test finds no significant difference at the 5% level in query counts or acceptance rates between GPT3 and InstructGPT.The two models therefore appear equally helpful according to these usage measures.
  • 3 HOW MUCH DO USERS ENGAGE WITH THE MODEL?: The paper summarizes each essay into key points and aligns each point to the sentence with the highest Rouge-L overlap.A key point is attributed to the model when more than half of its aligned sentence’s characters are model-generated.
  • 3 HOW MUCH DO USERS ENGAGE WITH THE MODEL?: The authors examine model contributions to key ideas because substantial model text may affect content similarity among users.This motivates testing whether model assistance increases homogenization.

4 DOES WRITING WITH LLMS RESULT IN MORE SIMILAR ESSAYS?

The paper measures essay homogenization as similarity among essays on the same topic and finds that InstructGPT, unlike GPT3, produces more similar content across users.

  • 4.1 METRIC: Homogenization is each essay’s average pairwise similarity to other essays written on the same topic.Corpus homogenization averages these essay-level scores; similarity is measured with Rouge-L and BertScore, where higher scores indicate more similar content.
  • 4.2 RESULTS: InstructGPT had the highest median topic homogenization on 7 out of 10 topics.The same trend was observed using BertScore.
  • 4.2 RESULTS: 0.1660 was the key-point Rouge-L corpus homogenization score for InstructGPT, compared with 0.1536 for Solo and 0.1578 for GPT3.InstructGPT’s score was significantly higher than both other groups at p < 0.05, and the result held across both similarity metrics and essay-level comparisons.
  • 4.2 RESULTS: GPT3 did not significantly increase homogenization relative to Solo despite contributing a similar amount of text and key points as InstructGPT.The difference between GPT3 and InstructGPT indicates that homogenization effects were not uniform across models.

5 DOES WRITING WITH LLMS REDUCE THE OVERALL DIVERSITY?

The paper evaluates diversity using lexical units and essay key points, finding that InstructGPT reduces both types of diversity through more repetitive and less varied model-contributed content.

  • 5.1 METRIC: Diversity is measured as the fraction of unique information units among all units in the essay collection.The paper evaluates n-grams for lexical diversity and clustered key points for content diversity.
  • 5.2 RESULTS: InstructGPT consistently produced lower lexical diversity than Solo and GPT3 across n-gram sizes from 1 through 5.GPT3 did not significantly decrease the diversity score.
  • 5.2 RESULTS: InstructGPT consistently exhibited lower key-point diversity across all selected clustering thresholds, significantly below both Solo and GPT3 at p < 0.05.The trend also held when BertScore was used as the distance measure.
  • 5.2 RESULTS: InstructGPT essays repeated common 5-grams more frequently despite similar unigram distributions.A concentration of probability mass in the head of the 5-gram distribution indicates greater repetition and lower text diversity.
  • 5.2 RESULTS: The diversity reduction measured with n-grams and key points also correlated with lower diversity in an information-theoretic sense.The paper additionally reports more frequent repetition of topical 5-grams with InstructGPT.

6 WHY DOES WRITING WITH INSTRUCTGPT REDUCE DIVERSITY?

The analysis attributes InstructGPT’s diversity reduction primarily to less diverse model-generated text, while user-written content remains largely stable across model settings.

  • InstructGPT generates less diverse text than GPT3, with average Rouge-L similarity scores of 0.20 versus 0.11.Higher similarity indicates lower generation diversity; the difference is significant at the 5% level.
  • InstructGPT increases repetition of higher-order n-grams, including 3-grams and 4-grams, in the co-written essays.The analysis also reports significantly different 4-gram and 5-gram usage across setups.
  • User-written text has similar 5-gram distributions across Solo, GPT3, and InstructGPT settings.This suggests that users’ lexical choices are largely unaffected by model assistance.
  • InstructGPT-contributed key points show higher homogenization, whereas user-contributed key points have similar scores with InstructGPT and GPT3.Homogenization scores are calculated separately for user- and model-attributed key points using Rouge-L.
  • The increased homogenization is mainly due to InstructGPT contributions, although user-contributed key points in co-written essays are not directly comparable to Solo key points.The latter are a subset of the original key points and therefore exhibit more variation across essays.

7 RELATED WORK

Related work examines human-AI collaborative writing, interactive text-generation evaluation, and the broader social impacts of language-model-mediated communication.

  • Human-AI collaborative writing: Earlier collaborative writing systems retrieved or generated suggestions, but users often needed to revise them before incorporation.The paper contrasts these systems with LLMs, which offer suggestions that are more readily incorporated.
  • Effect of human-AI co-writing: Recent studies examine how suggestion frequency affects writing efficiency, GPT3 collaboration affects vocabulary diversity and productivity, and user-model interactions shape writing.This work extends that research by formalizing content-diversity analysis in co-writing.
  • Evaluation of interactive text generation: Interactive text-generation evaluation includes reference-similarity metrics and distributional measures such as Mauve that expose quality-diversity trade-offs.The paper situates its diversity metrics within this broader evaluation literature.
  • Social impacts of LLMs: Social-impact research discusses algorithmic monoculture, language homogenization, predictive-keyboard effects, and LLM-mediated changes in user opinions.This paper investigates content diversity as one axis for evaluating general-purpose LLMs in co-writing settings.

8 CONCLUSION

The paper finds that InstructGPT makes users’ essays more similar and less diverse than essays written with GPT3 or without model help, primarily through less diverse model text.

  • Users writing with InstructGPT produce more similar content than users writing with GPT3 or without model help.This homogenization also reduces the overall diversity of content produced by many users.
  • The reduction in diversity is attributable to less diverse text contributed by InstructGPT, while user-contributed text remains largely consistent across settings.The conclusion frames content diversity as an additional axis for evaluating LLM impacts before deployment.
  • Human-feedback adaptation may improve instruction following while being accompanied by reduced content diversity.The paper connects this trade-off to concerns about preserving users’ voices where personal expression is desired.

REPRODUCIBILITY STATEMENT

The paper reports that its data and analysis materials will be released after the review period to support reproducibility.

  • Essays with character-level logs, model suggestions, and analysis scripts will be publicly released after the review period.The materials are intended to let readers replay writing sessions and reproduce the reported analyses.

ETHICS STATEMENT

The study recruited participants under an approved protocol, collected essays across three writing settings, and analyzed model-assisted writing with controlled procedures and statistical tests.

  • The study received Institutional Review Board approval and participant consent to share essays publicly after completion.
  • 38 native English-speaking Upwork participants each completed essay-writing tasks across the three settings, producing 100 essays per setting.Participants wrote on different topics across settings, with randomized order used to reduce opinion repetition.
  • The model-assisted conditions used davinci and text-davinci-003 with high-randomness decoding, including temperature 0.9 and frequency penalty 0.5.
  • The experiment used argumentative essays of around 300 words on open-ended New York Times Student Opinion topics.Participants were instructed to write three short passages expressing and supporting their opinions.
  • Participants using AI had to request at least five suggestions but could accept, edit, request again, or reject them.
  • Essay key points were generated with gpt-3.5-turbo, and permutation, chi-square, and independent-samples t-tests assessed diversity and homogenization differences.The analysis also reported basic essay statistics, including perplexity, length, and structural complexity.

C ADDITIONAL RESULTS

Additional analyses show that InstructGPT essays are more homogenized and less diverse across lexical, content, structural, and information-theoretic measures, while essay quality and length remain broadly similar.

  • Model-assisted essays contain fewer unique POS-n-grams and more repetition of frequent 4- and 5-POS-ngrams, with similar effects for GPT3 and InstructGPT.
  • Human ratings found no clear quality differences across setups on relevance, grammaticality, or discussion depth.The machine-written fraction in collaborative essays averaged approximately 32%-35%.
  • InstructGPT essays have significantly higher homogenization than Solo and GPT3 essays at both key-point and raw-essay levels using Rouge-L and BertScore.This pattern holds across the reported corpus-level comparisons and is statistically significant at p < 0.05.
  • InstructGPT essays concentrate probability mass on common 4-grams and 5-grams, whereas 1-, 2-, and 3-gram distributions are nearly identical across setups.
  • User-written n-gram distributions remain similar across settings, while InstructGPT contributes text with more common 3-, 4-, and 5-grams.
  • InstructGPT essays are least diverse across n-gram sizes and consistently show lower key-point diversity across similarity metrics and clustering thresholds.
  • InstructGPT essays are more compressible across LZMA, ZLIB, and GZIP, with statistically significant differences at p < 0.05.
  • InstructGPT presents less diverse suggestions than GPT3, and model-written document fraction shows a weak correlation with homogenization for InstructGPT.

D LIMITATIONS

The findings are bounded by the interaction interface, participant population, and limited set of OpenAI models used in the experiments.

  • The interface provided continuation suggestions, so further work is needed to test whether richer interaction or prompt engineering mitigates reduced diversity.
  • Because participants were recruited from a specific Upwork population, generalization to students learning to write or second-language speakers is unclear.
  • The experiments used two limited-access OpenAI models, and other models may behave differently because RLHF pipelines are highly customized.
Loading 2309.05196v3…