Source-linked AI summary
The Widespread Adoption of Large Language Model-Assisted Writing Across Society
Weixin Liang, Yaohui Zhang, Mihai Codreanu, Jiayu Wang, Hancheng Cao, James Zou
TL;DR
The paper addresses limited systematic evidence on LLM adoption across diverse writing domains by analyzing adoption patterns across consumer, firm, and institutional communications. It finds rapid post-ChatGPT uptake that stabilized later, indicating widespread LLM-assisted writing across society.
Problem
Systematic evidence remains limited on the patterns and extent of LLM adoption across diverse writing domains, because prior studies often covered single domains, used black-box detectors, or analyzed small datasets.
Method
The study conducts a large-scale, systematic analysis of LLM adoption across consumer complaints, corporate communications, job postings, and international organization press releases.
Results
LLM adoption rose rapidly after ChatGPT’s release and stabilized later, reaching about 18% in financial complaints, 24% in company press releases, up to 15% in young and small companies’ job postings, and 14% in international organizations.
Takeaways & Limitations
LLM-assisted writing is a pervasive reality across consumer, corporate, recruitment, and governmental communications, making its effects on content quality, creativity, credibility, and public trust important to understand.
Takeaways & Limitations
The estimates are lower bounds because heavily human-edited or highly human-like LLM-generated language may not be reliably detected, and the analysis primarily covers English-language content.
Abstract
from arXiv · showhide
The recent advances in large language models (LLMs) attracted significant public and policymaker interest in its adoption patterns. In this paper, we systematically analyze LLM-assisted writing across four domains-consumer complaints, corporate communications, job postings, and international organization press releases-from January 2022 to September 2024. Our dataset includes 687,241 consumer complaints, 537,413 corporate press releases, 304.3 million job postings, and 15,919 United Nations (UN) press releases. Using a robust population-level statistical framework, we find that LLM usage surged following the release of ChatGPT in November 2022. By late 2024, roughly 18% of financial consumer complaint text appears to be LLM-assisted, with adoption patterns spread broadly across regions and slightly higher in urban areas. For corporate press releases, up to 24% of the text is attributable to LLMs. In job postings, LLM-assisted writing accounts for just below 10% in small firms, and is even more common among younger firms. UN press releases also reflect this trend, with nearly 14% of content being generated or modified by LLMs. Although adoption climbed rapidly post-ChatGPT, growth appears to have stabilized by 2024, reflecting either saturation in LLM adoption or increasing subtlety of more advanced models. Our study shows the emergence of a new reality in which firms, consumers and even international organizations substantially rely on generative AI for communications.
Introduction
The paper addresses limited systematic evidence on LLM adoption across diverse writing domains by applying a transparent statistical framework to consumer, corporate, recruitment, and institutional communications. It finds rapid, broadly distributed adoption after ChatGPT’s release, with substantial usage across domains.
- Systematic evidence on LLM adoption remains limited across diverse writing domains, with prior studies often restricted to single domains, small datasets, or black-box detectors.
- The study uses a statistical framework designed to quantify LLM-modified content with greater robustness, transparency, and lower cost than commercial AI detectors.
- The analysis covers consumer complaints, corporate press releases, job postings, and United Nations press releases as distinct communication domains.
- By the end of the study period, estimated LLM-generated content reached about 18% in financial complaints, around 24% in company press releases, up to 15% in young and small firms’ job postings, and 14% in international organizations.
- Adoption patterns were broadly similar across domains, while organizational age and size emerged as important predictors, with smaller and younger firms showing higher utilization.
- These findings inform policymakers, business leaders, and researchers assessing AI integration and equitable, responsible deployment across sectors.
Results
Across consumer complaints, corporate communications, job postings, and UN press releases, LLM-related writing rose sharply after ChatGPT and generally stabilized by late 2023 or 2024. Adoption varied by geography, topic, firm age, and firm size, while detection and interpretation remain bounded by model sophistication and data limitations.
- Adoption rose rapidly after ChatGPT’s release and stabilized across all analyzed domains by mid to late 2023.
- Consumer complaints: Consumer-complaint text flagged as LLM-generated or substantially modified rose from a 1.5% false-positive baseline to 15.3% by August 2023 and 17.7% by August 2024.
- Corporate press releases: Newswire press releases peaked at 24.3% in December 2023 and stabilized at 23.8% through September 2024, while PRNewswire reached 16.4%.
- International organizations: UN press releases increased from 3.1% in Q1 2023 to 10.1% in Q3 2023 and 13.7% by Q3 2024.
- Consumer complaints: Highly urbanized areas reached 18.2% adoption versus 10.9% in non-highly urbanized areas, while lower-education areas reached 19.9% versus 17.4% in 2024Q3.
- Corporate press releases: Business & Money and Science & Tech press releases showed the strongest category increases, with Science & Tech reaching just below 17% by Q4 2023.
- Job postings: Small-company job postings leveled off around 5–10% AI-modified content, with engineering and sales each approaching 10%.
- Job postings: Firms founded after 2015 consistently showed the highest and fastest uptake, reaching 10–15% AI-modified text in some roles.
Discussion
LLM-assisted writing appears widespread across consumer, corporate, recruitment, and international-organization communications, with adoption surging after ChatGPT and stabilizing by late 2023. The study also highlights demographic variation, trade-offs in communication quality, and limits on detecting subtle or edited AI-generated text.
- Cross-domain adoption: LLM adoption surged after ChatGPT and stabilized by late 2023 across diverse writing domains.The authors suggest saturation, domain-specific barriers, or increasingly indistinguishable AI writing may explain the plateau.
- Consumer complaints: Consumer complaints showed higher adoption in highly urbanized areas, while lower educational attainment was associated with modestly higher use.The authors interpret this pattern as a possible departure from historical technology-diffusion trends, while noting that consumer-outcome effects require further study.
- Corporate communications: Corporate communications combined widespread, decelerating LLM integration with a trade-off between cost efficiency and authenticity.The discussion raises concerns that overreliance on automated writing could compromise nuance and credibility.
- Recruitment: Small firms, particularly those founded after 2015, exhibited the fastest adoption of LLM-generated job-posting content.The authors suggest possible effects on cost reduction, posting homogenization, applicant decisions, and hiring outcomes.
- International organizations: LLM-generated content also appeared in formal international-organization communications, extending adoption into traditionally cautious, high-stakes settings.The authors identify a similar cost-efficiency versus credibility trade-off in these communications.
- Limitations: Other models contribute to content generation, and shifts in user demographics or language usage could influence detection accuracy.The authors acknowledge that the study focuses on widely used models such as ChatGPT despite consistently low earlier-period false-positive rates.
- Limitations: The study cannot reliably detect heavily human-edited LLM text or outputs from models that closely imitate human writing, making estimates lower bounds of adoption.The analysis also focuses mainly on English-language content, potentially overlooking non-English adoption.
- Conclusion: The authors conclude that LLM writing is pervasive across consumer, corporate, recruitment, and governmental communications.They emphasize the need to understand effects on content quality, creativity, credibility, transparency, diversity, and public trust.
Supplementary Information
The supplementary information describes the datasets, sampling choices, and modeling framework used to quantify LLM-modified writing across communication domains. It also presents validation procedures and example prompts for compressing and expanding consumer complaints.
- Consumer complaint data: The consumer complaint dataset contains 687,241 narratives from January 2022 through August 2024.ZIP codes, RUCA codes, and local educational attainment support analysis of geographic and demographic heterogeneity.
- Job-posting data: The job-posting data contain 304,270,122 listings from January 2021 through October 2023 across eight occupational categories.Firm characteristics are linked to aggregated LinkedIn workforce data and defined using pre-ChatGPT information.
- Corporate press-release data: Corporate press releases were collected from PRNewswire, PRWeb, and Newswire, with analysis focused primarily on full English-language body text.Up to 537,413 releases were gathered from January 2022 through September 2024; robustness checks focused on PRNewswire and PRWeb because they provided sufficient post-ChatGPT volume.
- UN press-release data: United Nations press releases were collected from English-language websites of 97 country teams between January 2019 and September 2024.The analysis primarily used full article body text.
- Modeling framework: The statistical models use pre-ChatGPT corpora for fitting and data from January 2022 onward for validation and inference.Separate models were developed by LinkedIn job category and corporate distribution platform, while consumer complaints and UN releases each used one domain-level model.
- Validation: Validation mixed pre-ChatGPT records with known LLM-modified proportions ranging from 0% to 25% in 2.5% increments.Model estimates were compared with the known proportions to assess accuracy and calibration under temporal distribution shift.
- Prompt examples: Supplementary prompts illustrate summarizing a consumer complaint into a concise skeleton and expanding that skeleton into a coherent full text.The examples frame the two steps as outline construction followed by detailed paragraph generation.