Source-linked AI summary
Assessing the Value of Coooperation in Wikipedia
Dennis M. Wilkinson, Bernardo A. Huberman
TL;DR
The paper asks how Wikipedia develops at the level of individual articles and whether edit-based measures reflect article quality. It models edit accretion with a simple stochastic mechanism and compares edit activity across featured and other articles. The results show disproportionate editing of highly relevant or visible topics and a continued average association between article quality and the number of collaborators.
Problem
Earlier work did not explain Wikipedia’s development at the level of individual articles, while the value of edit-based quality metrics lacked justification.
Method
The paper models article edit accretion with a stochastic process and compares edits and contributors for featured versus other articles while controlling for visibility, relevance, and age.
Results
A small number of highly relevant or visible articles accrete a disproportionately large number of edits, and article quality continues to increase on average with collaborators.
Takeaways & Limitations
Wikipedia’s complex, decentralized editing produces a simple overall pattern consistent with collaborative improvement in article quality.
Takeaways & Limitations
Articles continue to accrete edits and evolve rather than reaching a steady state.
Abstract
from arXiv · showhide
Since its inception six years ago, the online encyclopedia Wikipedia has accumulated 6.40 million articles and 250 million edits, contributed in a predominantly undirected and haphazard fashion by 5.77 million unvetted volunteers. Despite the apparent lack of order, the 50 million edits by 4.8 million contributors to the 1.5 million articles in the English-language Wikipedia follow strong certain overall regularities. We show that the accretion of edits to an article is described by a simple stochastic mechanism, resulting in a heavy tail of highly visible articles with a large number of edits. We also demonstrate a crucial correlation between article quality and number of edits, which validates Wikipedia as a successful collaborative effort.
Introduction
Wikipedia’s rapid, decentralized growth raises questions about article quality and development, but its edits follow simple regularities. The paper links edit volume with article quality and relevance.
- Motivation: Wikipedia’s growth and article quality matter for evaluating it as a cooperative process and because of its widespread use.The encyclopedia expanded through contributions from millions of volunteers and became a major information source.
- Prior work: Prior work described Wikipedia as a complex system but did not explain development at the level of individual articles.Earlier studies examined network dynamics, article creation, editor roles, and edit distributions without proposing a mechanism for article-level development.
- Quality assessment: Existing quality metrics lacked justification, while combined metrics depended on arbitrary parameter choices and often omitted article popularity or relevance.The paper notes that quality assessment cannot reliably rest on a single metric and that popularity can affect edits, links, length, and images.
- Article growth: Wikipedia articles accrete edits through a simple stochastic mechanism in which new edits are a randomly varying percentage of previous edits.This mechanism is summarized as “edits beget edits.”
- Article growth: The resulting lognormal distribution contains a disproportionately large population of heavily edited articles, while article cohorts continue accumulating edits rather than reaching a steady state.The distribution parameters depend linearly on the age of the time slice, and the increase in µ with age indicates continued accretion.
- Edits and article quality: Featured articles have more edits and distinct editors than other articles, after controlling for article visibility, relevance, and age.Featured articles are selected by the Wikipedia community as “the best articles in Wikipedia.”
Article growth
Wikipedia article edits follow a simple stochastic, multiplicative accretion mechanism that produces lognormal edit-count distributions. This heavy-tailed process concentrates edits and visibility in a small set of articles, while edit counts are associated with article quality.
- Growth mechanism: Individual editing is highly variable, but the overall pattern of article edit accretion is well described by a simple stochastic mechanism.The model treats new edits as proportional on average to an article’s accumulated edits, with random fluctuations.
- Growth mechanism: Lognormal edit-count distributions arise because each article’s edit total changes multiplicatively through time under random activity.The model predicts the distribution in equation (2), with log-distribution parameters that vary linearly with article age.
- Empirical validation: 50.9% of 3,688 relevant time slices had p-values greater than 0.5 for the lognormal distribution.The analysis covered 50.0 million edits by 4.79 million non-robot contributors to 1.48 million English-language articles.
- Model qualification: The model’s independent-noise assumption is qualified by small positive autocorrelation in percentage edit increases over periods shorter than 20 to 30 days.The authors state that the resulting rate-parameter modification is small and omit it for simplicity.
- Visibility and concentration: A heavy tail means a small number of articles accumulate a disproportionately large number of edits, while most articles are infrequently edited and less visible.The modeled distribution retains a lognormal character when combined across article ages and Wikipedia’s growth.
- Quality and visibility: Edits correspond on average to increased article quality, and the multiplicative process creates a small body of high-quality, highly visible articles.The paper connects extensive editing by diverse contributors with article quality and visibility.
Edits and article quality
The paper tests whether editing and collaboration measures track article quality while controlling for article relevance and age. It finds strong associations between quality, edit volume, and distinct editors, although aggregate causality cannot be resolved.
- Controls: Article visibility or relevance and age must be controlled when comparing edit volumes across Wikipedia populations.The analysis groups articles by Google pagerank and normalizes log edit counts by the mean and variance for articles of the same age.
- Correlation: For all pageranks except 7, edits and distinct editors show a strong correlation with article quality.The anomalous pagerank-7 behavior disappears after accounting for article age.
- Method: Featured and nonfeatured populations were compared using an age-normalized edit-volume measure.The measure subtracts the age-specific mean log edit count and expresses the difference in units of the age-specific standard deviation.
- Correlation: The plots show a strong correlation between edits, distinct editors, and article quality, with the heavy tail consisting predominantly of high-quality articles.Pagerank, reportedly logarithmic, is more or less linearly related to the number of edits or editors.
- Causality: Articles continue to accrete edits as they age rather than reaching a steady state.This continuing evolution complicates attempts to determine whether editing causes quality, quality causes editing, or both processes contribute.
Conclusion
Despite unscheduled and nearly uncontrolled collaboration, Wikipedia editing follows a simple overall pattern. Highly relevant or visible topics receive disproportionately many edits, while quality increases on average with collaboration and editing.
- Conclusion: Wikipedia editing remains almost completely unsupervised despite contributions from diverse editors.The paper characterizes the process as unscheduled and virtually uncontrolled while still exhibiting a simple aggregate pattern.
- Conclusion: A small number of highly relevant or visible articles accrete a disproportionately large number of edits.These articles form the heavy tail of Wikipedia’s edit distribution.
- Conclusion: Article quality continues to increase on average as the number of collaborators and edits increases.The conclusion presents this relationship as evidence that Wikipedia’s large collaboration can improve article quality.
- Conclusion: Topics of high interest or relevance are naturally brought to the forefront of visibility and quality.This connects topic visibility with both disproportionate editing and higher observed quality.
Appendix: Methods
The study analyzes a large English-Wikipedia edit dataset after removing non-content pages and robot edits, then fits age-based edit distributions and controls comparisons by article visibility.
- Data: 55.3 million English-Wikipedia edits from January 2001 through November 2006 formed the raw dataset.The data included usernames or URLs, page titles, and timestamps.
- Data cleaning: Redirects, disambiguation pages, and robot edits were removed before analysis.Robot detection used the registered-robot list and improbably rapid successive edits; 5.23 million edits, or 9.5%, were eliminated.
- Data cleaning: Robot edits were identified using Wikipedia’s registered-robot list and unusually rapid successive edits.This procedure contributed to removing 5.23 million edits from the original dataset.
- Visibility controls: Articles with no Google pagerank, including some recent articles, were excluded from the pagerank analysis.A small number of articles were also omitted because rare foreign characters caused technical title-processing difficulties.