Source-linked AI summary
Mapping the Increasing Use of LLMs in Scientific Papers
Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Diyi Yang, Christopher Potts, Christopher D Manning, James Y. Zou
TL;DR
The paper addresses the lack of a precise measure of how much scientific writing is substantially modified by LLMs. It analyzes 950,965 papers with a population-level quantification framework and finds increasing use, strongest in Computer Science, alongside aggregate associations with preprinting, field crowding, and shorter papers.
Problem
The study addresses the lack of a precise measure of the proportion of scientific writing substantially modified by LLMs, a gap relevant to understanding academic publishing at scale.
Method
The authors apply a population-level distributional GPT quantification framework to abstracts and introductions from 950,965 papers across arXiv, bioRxiv, and 15 Nature portfolio journals.
Results
LLM-modified content increased steadily, with the fastest and largest growth in Computer Science, reaching 17.5% in abstracts by February 2024.
Takeaways & Limitations
Higher LLM-modification was associated with more frequent first-author preprinting, more crowded research areas, and shorter papers.
Takeaways & Limitations
The study focuses on ChatGPT, and author-population or other language-use changes could still affect estimate accuracy.
Abstract
from arXiv · showhide
Scientific publishing lays the foundation of science by disseminating research findings, fostering collaboration, encouraging reproducibility, and ensuring that scientific knowledge is accessible, verifiable, and built upon over time. Recently, there has been immense speculation about how many people are using large language models (LLMs) like ChatGPT in their academic writing, and to what extent this tool might have an effect on global scientific practices. However, we lack a precise measure of the proportion of academic writing substantially modified or produced by LLMs. To address this gap, we conduct the first systematic, large-scale analysis across 950,965 papers published between January 2020 and February 2024 on the arXiv, bioRxiv, and Nature portfolio journals, using a population-level statistical framework to measure the prevalence of LLM-modified content over time. Our statistical estimation operates on the corpus level and is more robust than inference on individual instances. Our findings reveal a steady increase in LLM usage, with the largest and fastest growth observed in Computer Science papers (up to 17.5%). In comparison, Mathematics papers and the Nature portfolio showed the least LLM modification (up to 6.3%). Moreover, at an aggregate level, our analysis reveals that higher levels of LLM-modification are associated with papers whose first authors post preprints more frequently, papers in more crowded research areas, and papers of shorter lengths. Our findings suggests that LLMs are being broadly used in scientific writings.
1 Introduction
The paper develops a population-level approach to estimate LLM-modified academic writing and applies it across venues and disciplines. It reports rising use, especially in Computer Science, alongside aggregate associations with author preprint frequency, field crowding, and paper length.
- Motivation: Individual instances of LLM-modified writing are difficult to detect, motivating corpus-level analysis of broader structural and linguistic shifts.The paper frames large-scale measurement as distinct from treating LLM use as an individual-level rule violation.
- Motivation: Institutional policies restricting LLM-generated content make measuring its prevalence relevant to concerns about accuracy, plagiarism, anonymity, and ownership.
- Contribution: The study defines LLM-modified text as content substantially updated by ChatGPT beyond basic spelling and grammatical edits.Examples include summarizing existing writing or generating prose from outlines.
- Contribution: The analysis covers abstracts and introductions from arXiv, bioRxiv, and 15 Nature portfolio journals using a population-level distributional GPT quantification framework.
- Results: 17.5%: Computer Science abstracts reached this estimated LLM-modified fraction by February 2024, the largest and fastest growth reported.The paper also reports 15.3% for Computer Science introductions, while Mathematics and the Nature portfolio showed the least increase.
- Results: Higher LLM-modification was associated with more frequent first-author preprinting, shorter papers, and greater similarity among papers in crowded research areas.The paper presents these relationships at an aggregate level and notes that similarity may reflect either LLM use or more similar research contexts.
2 Related Work
The related work describes two broad families of LLM-text detectors: zero-shot methods based on statistical signatures and training-based binary classifiers.
- Detection approaches: Zero-shot detectors identify machine-generated text using statistical signatures characteristic of LLM-produced content.
- Detection approaches: Training-based detectors fine-tune language models to classify text as human-written or LLM-modified.
- Detection approaches: The paper situates its approach within existing methods for detecting LLM-modified text.
3 Background: the distributional LLM quantification framework
The framework estimates the fraction of LLM-modified content in a corpus by modeling human and LLM text distributions and fitting their mixture to observed sentences. It uses counterfactual LLM-generated training data and full-vocabulary token statistics to improve realism and estimation stability.
- Framework overview: The framework adapts distributional LLM quantification to estimate AI-modified academic writing through a sequence of modeling, data-generation, and inference steps.
- Problem formulation: The observed corpus is modeled as a mixture of human-written and LLM-modified documents, with α representing the fraction of AI-modified documents.
- Parameterization: Human and LLM-modified document distributions are parameterized by token-occurrence probabilities for a selected vocabulary.
- Estimation and inference: The method estimates token probabilities from known human-written and LLM-modified collections before fitting the observed corpus mixture.
- Training-data generation: LLM training paragraphs are generated by first summarizing human-written text into an outline, then producing a full paragraph from that outline.This counterfactual procedure preserves much of the source content while simulating outline-based LLM assistance.
- Vocabulary choice: Using the full vocabulary reduces vocabulary-selection bias and yields more sample-efficient, stable estimates with smaller bootstrap confidence intervals.
4 Implementation and Validations
The study samples papers across three scholarly sources, fits separate models for abstracts and introductions, and evaluates calibration under temporal distribution shift. Validation uses pre-ChatGPT papers with controlled LLM-modification proportions and reports low population-level prediction error.
- Data collection: Up to 2,000 papers per month were randomly sampled from arXiv, bioRxiv, and 15 Nature portfolio journals from January 2020 through February 2024.
- Data collection: The study focuses on introductions for main-text analysis because this section occurs most consistently across diverse paper categories.
- Model fitting: Models were fitted with 2020 papers and validated or applied from January 2021 onward, separately for abstracts and introductions in each major category.
- Evaluation: Validation used 3,000 pre-ChatGPT papers from January 1 through November 29, 2022, with α ranging from 0% to 25% in 5% increments.
- Evaluation: Less than 3.5%: prediction error remained below this level across tested population-level ground-truth α values and academic-writing venues.
- Evaluation: Figure 3 compares ground-truth α on the x-axis with estimated α on the y-axis across abstract and introduction validation panels.The panels use a temporally separated validation split, and error bars show 95% bootstrap confidence intervals.
5 Main Results and Findings
LLM-modified content increased steadily, with the largest and fastest growth in Computer Science. Higher modification levels were also associated with more frequent first-author preprint posting, greater paper similarity to peers, and shorter papers.
- Temporal Trends in AI-Modified Academic Writing: 17.5% of Computer Science abstracts and 15.5% of introductions were estimated to contain LLM-modified content by February 2024.Mathematics reached 4.9% for abstracts and 3.9% for introductions, while the Nature portfolio reached 6.3% and 4.3%, respectively.
- Temporal Trends in AI-Modified Academic Writing: November 2022 estimates were 2.3% for Computer Science, 2.9% for Electrical Engineering and Systems Science, 2.4% for Mathematics, and 3.1% for the Nature portfolio.These estimates provided a pre-ChatGPT reference point because ChatGPT launched on November 30, 2022.
- First-Author Preprint Posting Frequency: 19.3% versus 15.6% of abstract sentences were LLM-modified for papers whose first authors posted ≥3 versus ≤2 preprints in 2023.The corresponding introduction estimates were 16.9% and 13.7%, and the trend held across cs.CV, cs.LG, and cs.CL.
- Paper Similarity and LLM Usage: 22.2% versus 14.7% of abstract sentences were LLM-modified in papers more versus less similar to their closest peer by February 2024.The similarity groups were defined by below- versus above-median nearest-neighbor embedding distance, and the trend held across three Computer Science subcategories.
- Paper Length and AI Usage: 17.7% versus 13.6% of abstract sentences were LLM-modified in shorter versus longer papers by February 2024.Papers were split at 5,000 full-text words including appendices; the difference held in cs.CV and cs.LG but not cs.CL, possibly because of limited sample size.
6 Discussion
LLM-modified content increased sharply in scientific writing, particularly in Computer Science, and was associated with preprint frequency, research-area crowding, and paper length. These patterns may reflect competitive pressures and raise questions about scientific independence and accuracy.
- 6 Discussion: LLM-modified content increased sharply beginning about five months after ChatGPT’s release, with the fastest growth in Computer Science.The authors suggest familiarity with and access to LLMs, along with pressure to publish quickly, may partially explain this pattern.
- 6 Discussion: Higher LLM-modification was associated with authors who posted preprints more frequently, papers in more crowded research areas, and shorter papers.The authors interpret these associations as possible indicators of competitive research environments and pressure to publish quickly.
- 6 Discussion: Private-company ownership of widely used LLMs could pose risks to the security and independence of scientific practice.The authors call for further study and discussion of transparent, epistemically diverse, accurate, and independent scientific publishing.
- 6 Discussion: The observed relationships are correlations that may be affected by author-population changes, language-use shifts, research topics, and the omission of non-ChatGPT models.The authors identify more causal studies as an important direction for future work.
A Estimated Fraction of LLM-Modified Sentences in Introductions
The study uses prompts to simulate LLM-assisted academic writing and examines LLM-modified sentence fractions in introductions over time. The introduction analysis is designed to provide a consistent cross-venue comparison and agrees with abstract results.
- A Estimated Fraction of LLM-Modified Sentences in Introductions: The introduction analysis estimates the fraction (α) of LLM-modified sentences across academic writing venues over time.Introductions were selected because they occurred most consistently across paper categories.
- A Estimated Fraction of LLM-Modified Sentences in Introductions: Introduction results were consistent with the abstract results, while bioRxiv introductions were excluded because bulk PDF downloads were unavailable.Error bars represent 95% bootstrap confidence intervals.
- A Estimated Fraction of LLM-Modified Sentences in Introductions: The study’s prompts include proofreading, summarizing a paragraph into a skeleton, and expanding that skeleton into full text.These prompts simulate stages of LLM-assisted writing from concise ideas to articulated academic prose.
C Additional Information on Implementation and Validations
The study combines sampled papers from arXiv, bioRxiv, and Nature journals with ChatGPT-focused training-data generation and specified decoding settings. Its implementation choices prioritize broad coverage and a representative analysis of LLM-assisted writing.
- C Additional Information on Implementation and Validations: The dataset was collected from arXiv and bioRxiv APIs and Nature portfolio webpages, using monthly sampling across major arXiv categories.When fewer than 2,000 papers were available for a month, all available papers were included.
- C Additional Information on Implementation and Validations: GPT-3.5-turbo-0125 generated the training data and was not used elsewhere in the study.The model was trained on data available through September 2021.
- C Additional Information on Implementation and Validations: ChatGPT was the study’s focus because it accounted for 76% of global internet traffic in the generative-AI category in early 2024.The authors use this market share to motivate ChatGPT’s relevance to their investigation.
- C Additional Information on Implementation and Validations: GPT-3.5 was selected partly for free availability, which the authors argue lowers the entry barrier and broadens captured usage patterns.Prior work is cited for the framework’s robustness and generalizability to other LLMs.
- C Additional Information on Implementation and Validations: Decoding used temperature 1.0, maximum length 2048 tokens, Top P 1.0, zero frequency and presence penalties, and no configured stop sequences.These settings specify how the training examples were generated.
D Word Frequency Shift in arXiv Computer Science introductions
The supplementary figures examine word-frequency shifts and whether associations between LLM usage and author activity, paper similarity, and paper length persist across Computer Science sub-categories. Their subgroup definitions and uncertainty estimates are explicitly specified.
- D Word Frequency Shift in arXiv Computer Science introductions: Word-frequency trends for realm, intricate, showcasing, and pivotal in sampled arXiv Computer Science introductions are shown over time.The analysis excludes 2010–2020 because parsing the full text from many arXiv papers was computationally complex.
- D Word Frequency Shift in arXiv Computer Science introductions: Preprint frequency is compared across cs.CV, cs.LG, and cs.CL using groups with ≤2 versus ≥3 first-authored preprints in a year.Error bars indicate 95% bootstrap confidence intervals.
- D Word Frequency Shift in arXiv Computer Science introductions: Paper similarity is compared within cs.CV, cs.LG, and cs.CL using below- versus above-median embedding distance to the closest peer.The grouping distinguishes papers more similar to their closest peer from papers less similar to that peer.
- D Word Frequency Shift in arXiv Computer Science introductions: Paper length is compared below and above 5,000 words, with the relationship holding for cs.CV and cs.LG but not cs.CL.The cs.CL result may reflect limited sample size because only a subset of PDFs was parsed.
F Proofreading Results on arXiv data
Figure 15 examines whether proofreading-related LLM edits change estimated LLM-modified content across arXiv categories. The analysis finds a slight increase, supporting robustness to minor edits.
- Figure 15 reports a slight increase in estimated LLM-modified content after LLM proofreading.The comparison spans different arXiv main categories.
- The proofreading analysis tests robustness to minor LLM-generated text edits.The passage identifies simple proofreading as an example of such edits.
- The analysis used 1,000 abstracts from each arXiv main category.The abstracts were randomly sampled from the stated study period.
G Extended Related Work
The extended related work reviews individual-document detection through zero-shot and training-based methods, alongside watermarking approaches. It also contrasts these methods with a model-independent population-level framework and discusses implications for LLM pretraining data.
- Zero-shot LLM detection: Zero-shot detection uses statistical signatures such as entropy, rare n-grams, perplexity, and probability curvature to identify AI-modified text.These methods target detection within individual documents and may require access to LLM internals.
- Training-based LLM detection: Training-based detection classifies human and AI-modified text using models trained or fine-tuned on labeled examples.The reviewed approaches include classifiers, Siamese networks, and contrastive or adversarial learning.
- Training-based LLM detection: Training-based detectors can overfit, remain vulnerable to adversarial attacks, and be biased against non-dominant language varieties.The passage also notes that accurate instance-level detection has been theoretically questioned.
- LLM watermarking: Watermarking embeds an imperceptible signal during generation so a detector with the model owner’s secret key can retrieve it.Modern strategies integrate watermarks into language-model decoding.
- LLM watermarking: Unlike watermarking, Liang et al.’s framework monitors AI-modified content without model-owner intervention.This distinguishes it from methods requiring the service owner to implant a watermark during generation.
- Implications for LLM Pretraining Data Quality: The paper connects increasing AI-modified content in academic papers with concerns about the quality of LLM pretraining data.arXiv contributes approximately 2.5% of Llama’s data, 12% of RedPajama’s, and 8.96% of the Pile’s.