Source-linked AI summary
LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities
Moyan Zhou, Soobin Cho, Loren Terveen
TL;DR
Wikipedia participation requires more than fluent text: editors must meet epistemic and social needs, while community norms for LLM use remain unsettled. Through interviews with 16 editors, the study finds expertise-based differences in contributions and community responses, and argues that LLMs can disrupt newcomers’ learning pathways.
Problem
LLMs produce fluent content, but Wikipedia participation also requires epistemic and social alignment, and the community had not reached consensus on LLM norms.
Source: §1 · pp. 1–2Method
The researchers interviewed 16 Wikipedia editors who had used LLMs and asked about their use, contributions, and other editors’ responses.
Source: §6 · p. 15Results
Editors’ expertise shaped an LLM-related participation divide: newcomers faced interrupted learning pathways and more central editorial judgments, while experienced editors’ contributions and community responses differed.
Source: §1 · p. 2, §6 · p. 15Takeaways & Limitations
The findings challenge existing models of newcomer involvement and motivate LLM designs that scaffold learning, teach norms, and support editors across expertise levels.
Source: §1 · p. 2, §5.2.2 · p. 14The qualitative study did not quantify how prevalent the observed phenomena were.
Source: §5.3 · p. 15
Abstract
from arXiv · showhide
Large language models (LLMs) are reshaping knowledge production as community members increasingly incorporate them into their contribution workflows. However, participating in knowledge communities involves more than just contributing content - it is also a deeply social process. While communities must carefully consider appropriate and responsible LLM integration, the absence of concrete norms has left individual editors to experiment and navigate LLM use on their own. Understanding how LLMs influence community participation is therefore critical in shaping future norms and supporting effective adoption. To address this gap, we investigated Wikipedia, one of the largest knowledge production communities, to understand 1) how LLMs influence the ways editors contribute content, 2) what strategies editors leverage to align LLM outputs with community norms, and 3) how other editors in the community respond to LLM-assisted contributions. Through interviews with 16 Wikipedia editors who had used LLMs for their edits, we found that 1) LLMs affected the content contributions for experienced and new editors differently; 2) aligning LLM outputs with community norms required tacit knowledge that often challenged newcomers; and 3) as a result, other editors responded to LLM-assisted edits differently depending on the editors' expertise level. Based on these findings, we challenge existing models of newcomer involvement and propose design implications for LLMs that support community engagement through scaffolding, teaching, and context awareness.
1 Introduction
LLMs make knowledge production easier, but participation also requires epistemic judgment and social alignment with community norms. This study examines how LLMs affect Wikipedia editors’ contributions, norm alignment, and interactions, finding that effects differ by expertise.
1 Introduction
- Knowledge-community participation involves factuality, balance, norm alignment, and engagement, needs that fluent LLM-generated text alone does not meet.Wikipedia had not developed consensus or norms around LLM use, leaving editors to interpret and navigate responsible use individually.Source: §1 · pp. 1–2
- The study asks how LLMs influence editors’ contributions, how editors align LLM-assisted contributions with norms, and how other editors respond.The questions address the contribution, norm-alignment, and community-response dimensions of participation.Source: §1 · p. 2
- Interviews with 16 editors found that experienced editors expanded their contributions and aligned LLM outputs with community norms, while newcomers faced higher participation demands and rejection.The study frames this as an expertise-based divide in how editors use LLMs and how other members respond.Source: §1 · p. 2
- LLMs can lower entry barriers yet increase the demands of participation, challenging newcomers’ gradual learning pathways and pushing them toward editorial judgments requiring central wiki skills.The authors propose scaffolding, teaching, and context-aware tools to support learning and participation across expertise levels.Source: §1 · p. 2
2 Related Work
The related work situates LLM-assisted writing within Wikipedia’s content, community, and norm systems. It contrasts LLMs, which require editors to make normative judgments, with tools that align with or reinforce community norms.
2.1 Human AI collaboration in writing
- Prior research covers generative AI across writing domains and studies tools for academic writing, including idea generation, literature reviews, and drafting workflows.Users may treat these tools as idea generators or collaborators and engage differently depending on the writing task.Source: §2.1 · p. 3
2.2 Knowledge production in Wikipedia
- Wikipedia participation includes contributing content, engaging with other editors, and enforcing norms.These three aspects frame the paper’s account of knowledge production in the community.Source: §2.2 · p. 3, §2.2.3 · p. 4
2.2.1 Content.
- Wikipedia content work includes meaning-preserving and meaning-changing edits, multilingual contributions, and efforts to bridge language and coverage gaps.Prior taxonomies distinguish edits such as paraphrasing from insertions or deletions.Source: §2.2.1 · pp. 3–4
2.2.2 Community.
- As editors gain experience, participation often shifts from individual edits toward community coordination, consensus-building, and conflict resolution.Editors use talk pages and WikiProjects for coordination and social interaction.Source: §2.2.2 · p. 4
2.2.3 Policies and Guidelines.
- Wikipedia policies and guidelines define content standards and expected behavior, supporting collaboration, quality, credibility, and coordination.They also help editors seek consensus and function as boundary objects in daily work.Source: §2.2.3 · p. 4
2.3 Tool-assisted editing in Wikipedia
- Wikipedia’s editing tools include bots and newer AI/ML systems designed to automate tasks, recommend edits, evaluate quality, or improve content.The section reviews semi-automated bots and more recent AI/ML tools.Source: §2.3 · p. 4, §2.3.1 · p. 5
2.3.1 Semi-automated bots.
- Traditional bots automate repetitive tasks and can support editing, but vandal-fighting bots may revert good-faith newcomers’ edits and harm retention.Editors also work alongside bots by checking information and authorizing actions; such tools are generally accepted and can act as social actors.Source: §2.3.1 · pp. 4–5
- Unlike traditional tools that conform to Wikipedia norms, LLMs bypass policies and guidelines and require editors to make normative decisions, challenging newcomer participation.The paper builds on this contrast to identify a distinct interaction pattern for LLM use.Source: §2.3.1 · p. 5
3 Methods
The researchers conducted semi-structured interviews with Wikipedia editors who had used LLMs and analyzed the material through inductive thematic analysis. Recruitment, interview questions, and coding were designed to capture contribution practices, LLM experiences, and future collaboration views.
3 Methods
- The researchers collaboratively designed a semi-structured protocol covering contribution context, LLM use in Wikipedia, and visions for human-AI collaboration.They piloted and refined the questions; pilot data were excluded from analysis.Source: §3 · p. 5
3.1 Recruitment and Participants
- The team recruited editors through Wikimedia project posts, direct invitations, and snowball referrals, then conducted 16 interviews, including five by email.They observed data saturation after interview 13 and stopped recruitment at 16.Source: §3.1 · p. 5
3.2 Interview Procedure
- Interviews covered participants’ typical contribution workflows, a specific LLM-use case, and their visions for future human-AI collaboration.Participants could share experiences freely, with follow-up questions from the interviewer.Source: §3.2 · p. 6
3.3 Data Analysis
- The authors used inductive thematic analysis, open-coded interviews, and collaboratively grouped codes into themes and subcategories.They established a shared coding standard on three interviews, coded the rest asynchronously, and reached agreement on themes.Source: §3.3 · p. 7
4 Results
The results show that expertise shapes how LLMs affect Wikipedia participation. Experienced editors used LLMs to extend their work and confidence, while the section details several ways they explored new topics, gained perspectives, and felt more capable in unfamiliar areas.
4 Results
- LLM use influenced content contributions, norm enforcement, and other members’ engagement, producing a participation divide shaped by editor expertise.The results section also catalogs reported LLM use cases, distinguishing generation from refinement based on whether content existed before interaction.Source: §4 · p. 7, Table 2 · p. 7
4.1 RQ1: Contributing content
- Experience mediated LLMs’ effects on contribution: experienced editors extended their cognitive capacities, while newcomers relied on LLMs to fill knowledge gaps and reduce entry barriers.For newcomers, that reliance came with increased demands to assess generated content.Source: §4.1 · pp. 7–8
4.1.1 Experienced editors venture into new topics and tasks.
- LLMs introduced experienced editors to unfamiliar topics and tasks, including article sections and code troubleshooting they might not have pursued alone.Editors checked factual information before adding it, and LLM suggestions could surface packages or perspectives they had not considered.Source: §4.1.1 · p. 8
4.1.2 Experienced editors are exposed to new perspectives.
- LLMs offered experienced editors unexpected perspectives on their writing, including suggestions about what readers might find interesting and pluralistic reader-like opinions.Editors used these perspectives to reconsider and evaluate their own content.Source: §4.1.2 · p. 8
4.1.3 Experienced editors are more confident in unfamiliar areas.
- LLM support increased experienced editors’ confidence and satisfaction, including when they worked in less familiar areas or used LLMs to simplify complex material.Editors also described using LLMs as an additional check for overlooked issues and feeling satisfied with work sooner.Source: §4.1.3 · p. 8, §4.1.2 · p. 8
I’ve written earlier...thus allow me to go to bed earlier."
LLMs were associated with improved contribution quality for experienced editors, while helping newcomers overcome entry barriers and build knowledge about Wikipedia editing.
4.1.4 LLMs enhance contribution quality for experienced editors.
- Experienced editors reported improved contribution accuracy and fewer corrections by other editors after using LLMs.P09 described improved accuracy on less familiar topics; P03 said fewer editors had to clean up his edits.Source: §4.1.4 · p. 9
4.1.5 LLMs lower entry barriers for newcomers.
- LLMs encouraged newcomers to edit by helping with information access and English proofreading, lowering barriers to contributing.P14 used an LLM before traveling to a library for information; P01 used ChatGPT to proofread because English was not his native language.Source: §4.1.5 · p. 9
- Newcomers also used LLMs to learn about topics and Wikipedia conventions, including citations and notability.Participants described using LLMs as research platforms, mentors, and guides for getting started with article creation.Source: §4.1.5 · p. 9
4.2 RQ2: Conforming to community norms
Editors navigated unclear LLM norms by applying Wikipedia policies and exercising editorial judgment. Aligning outputs with those norms required expertise that newcomers often had not yet developed.
4.2.1 LLMs fail to align with core Wikipedia content policies.
- LLMs could conflict with Wikipedia’s Neutral Point of View, Verifiability, and No Original Research policies through promotional language, unreliable sources, and hallucinated claims.Participants also reported fabricated references, especially for obscure content.Source: §4.2.1 · p. 10
4.2.2 Participants’ strategies to make editorial judgements.
- Editors acted as gatekeepers by evaluating LLM output, verifying its claims, and modifying it to meet Wikipedia’s tone, style, and format.Evaluation assessed readability and coherence; verification checked factual support, while modification included neutralizing tone and fixing wiki links or wikitext.Source: §4.2.2 · p. 10
4.2.3 Newcomers as gatekeepers?
- Newcomers often lacked the Wikipedia knowledge needed to judge whether LLM-generated material was appropriate, making gatekeeping challenging.P15 described struggling to identify what content to filter until another editor pointed out what Wikipedia did not want.Source: §4.2.3 · p. 11
4.3 RQ3: Community engagement
Other editors’ responses to LLM-assisted contributions differed by editor expertise: newcomers faced rejection, while experienced editors often received approval. Community responses also reflected sensitivity and uncertainty about what LLM use entails.
4.3 RQ3: Community engagement
- Other editors’ responses differed by expertise, praising or accepting experienced editors’ contributions while calling out or rejecting newcomers’ edits.The paper links this difference to whether the content conformed to standards and whether others perceived it as AI-generated.Source: §4.3 · p. 11
4.3.1 Other editors responded to newcomers with rejections.
- Newcomers reported heightened scrutiny, with some LLM-assisted contributions rejected as machine-generated, promotional, or low quality.Participants described drafts being flagged and removed, and one newcomer removed LLM-generated content after another editor identified its use.Source: §4.3.1 · p. 11
4.3.2 Other editors responded to experienced editors with approval.
- Experienced editors often received positive feedback or saw little reaction to their LLM-assisted edits, though some were falsely accused of using LLMs.Participants associated limited detection with extensive post-editing, while one reported being wrongly called out.Source: §4.3.2 · pp. 11–12
4.3.3 Overall, the community remains sensitive and confused about LLM uses.
- Unclear definitions of LLM use contributed to mistrust, including criticism of edits perceived as AI-generated even when factually accurate.The accounts suggest community members care about content origin as well as content quality.Source: §4.3.2 · p. 12, §4.3.3 · p. 12
5.1 The paradox of participation
LLMs create a paradox of participation: they lower entry barriers while increasing the demands on newcomers, contributing to a participation divide shaped by expertise.
5.1 The paradox of participation
- LLMs lower barriers while increasing demands on newcomers, challenging expectations that participation progresses through gradual, socially situated learning.The authors describe a divide in which experienced editors thrive while newcomers are marginalized.Source: §5.1 · p. 12
5.1.1 Interrupted learning pathways for newcomers (Situated learning).
- LLMs let newcomers take on complex tasks using linguistic support, contextual suggestions, and source search, but can bypass gradual learning and opportunities to internalize Wikipedia norms.The authors characterize this as accelerated participation that shortcuts social situated learning.Source: §5.1.1 · pp. 12–13
5.1.2 A shift from peripheral tasks to editorial judgment (LPP).
- Instead of beginning with lower-risk contributions, newcomers must evaluate, verify, and modify LLM output, requiring editorial judgment and understanding of Wikipedia’s norms.The authors say these responsibilities assume epistemic and social maturity and may arise with little feedback or guidance.Source: §5.1.2 · p. 13
5.1.3 A new participation divide mediated by expertise (Second-level digital divide).
- Experienced editors can critically adapt LLM output using their knowledge of Wikipedia policies, while the participation divide stems from unequal ability to use LLMs effectively, not unequal access.The authors relate this skills-and-knowledge disparity to the second-level digital divide.Source: §5.1.3 · p. 13
5.2 Design implications
LLMs can expand participation and lower technical barriers, but access alone does not ensure newcomers participate legitimately. The proposed assistants should scaffold work, teach norms, and adapt to editor expertise.
5.2 Design implications
- LLMs can empower newcomers to contribute more while demanding more of them, and experienced editors can expand participation across domains.The authors describe this as a participation paradox: increased access does not guarantee better participation.Source: §5.2 · pp. 13–14
5.2.1 Scaffold participation through incremental guidance.
- Rather than generating a complete article from one prompt, assistants should break complex work into steps that let newcomers learn by doing.Suggested steps include finding sources, summarizing content, and structuring the article.Source: §5.2.1 · p. 14
5.2.2 Teach community norms through interactions.
- Assistants should embed normative feedback that helps newcomers assess whether contributions align with community standards.They can flag problematic passages, show acceptable and unacceptable examples, and ask guided questions.Source: §5.2.2 · p. 14
5.2.3 Be aware of and personalize based on user expertise.
- Responses should vary with expertise: newcomers may need more guardrails, while experienced editors may benefit from direct outputs and greater freedom.For newcomers, walking through each step can support learning by doing; experienced editors may use the LLM as a co-writer.Source: §5.2.3 · p. 14, §5.2.2 · p. 14
- The authors leave open whether newcomers bypass peripheral tasks because of LLMs or would begin with complex tasks regardless.They identify this as an emerging dynamic for future research.Source: §5.2.3 · p. 14
5.3 Limitations and future work
The study offers qualitative insight into editors’ experiences, but its design limits what can be concluded about prevalence and whose perspectives are represented.
5.3 Limitations and future work
- The qualitative approach captured nuanced experiences but did not quantify how prevalent the observed phenomena are.The authors suggest using their use cases to develop surveys assessing generalizability and prevalence.Source: §5.3 · p. 15
- Because the study included only editors who had used LLMs, it may overrepresent confidence in adoption and underrepresent deliberate nonusers.The authors also recommend studying editors who initially used LLMs but later stopped.Source: §5.3 · p. 15
6 Conclusion
Interviews with 16 Wikipedia editors who used LLMs revealed an expertise-based participation divide. The study describes a newcomer participation paradox and proposes design implications to mitigate the gap.
6 Conclusion
- Interviews with 16 Wikipedia editors who had used LLMs found an expertise-based divide in content contribution, norm enforcement, and other editors’ engagement.The study examined editors’ perceptions, adoption, interactions with LLMs, and responses to LLM-assisted edits.Source: §6 · p. 15
- For newcomers, LLMs interrupt learning pathways and shift peripheral tasks into central tasks requiring editorial judgment.The conclusion characterizes this as a participation paradox.Source: §6 · p. 15
- Experienced editors can enhance participation by exploring new topics, gaining multiple perspectives, and increasing confidence.Source: §6 · p. 15
- The proposed design implications include scaffolding complex tasks, educating users about community norms, and accounting for expertise as context.The authors frame user-experience research as important for equitable AI integration and community collaboration.Source: §6 · p. 15