Source-linked AI summary
The Wisdom of Polarized Crowds
Feng Shi, Misha Teplitskiy, Eamon Duede, James Evans
TL;DR
The paper asks whether self-selected teams of politically diverse individuals produce better knowledge despite the conflict associated with polarization. Using millions of Wikipedia edits, editor-alignment measures, survey validation, and talk-page analysis, it finds that balanced polarized teams create higher-quality articles and engage in more focused, linguistically diverse debate. The authors also discuss institutional designs that may help channel such conflict productively.
Problem
It is unclear whether politically diverse teams create higher- or lower-quality outcomes because political diversity may provide fresh perspectives while undermining cooperation.
Method
The study analyzes millions of Wikipedia edits, estimates editors’ political alignments from conservative-versus-liberal article contributions, validates the measure by survey, and examines article quality and talk-page interactions.
Results
Balanced politically polarized Wikipedia teams produce higher-quality articles than homogeneous teams, with effects observed in Political, Social Issues, and Science articles and associated with focused, robust, comprehensive debate.
Takeaways & Limitations
For contested knowledge, platforms may benefit from balanced political perspectives combined with enforceable policies and guidelines that discipline motivated conflict.
Takeaways & Limitations
Because the study is observational and editors self-select collaborations, it cannot establish that randomly assigned polarized teams would outperform homogeneous teams.
Abstract
from arXiv · showhide
As political polarization in the United States continues to rise, the question of whether polarized individuals can fruitfully cooperate becomes pressing. Although diversity of individual perspectives typically leads to superior team performance on complex tasks, strong political perspectives have been associated with conflict, misinformation and a reluctance to engage with people and perspectives beyond one's echo chamber. It is unclear whether self-selected teams of politically diverse individuals will create higher or lower quality outcomes. In this paper, we explore the effect of team political composition on performance through analysis of millions of edits to Wikipedia's Political, Social Issues, and Science articles. We measure editors' political alignments by their contributions to conservative versus liberal articles. A survey of editors validates that those who primarily edit liberal articles identify more strongly with the Democratic party and those who edit conservative ones with the Republican party. Our analysis then reveals that polarized teams---those consisting of a balanced set of politically diverse editors---create articles of higher quality than politically homogeneous teams. The effect appears most strongly in Wikipedia's Political articles, but is also observed in Social Issues and even Science articles. Analysis of article "talk pages" reveals that politically polarized teams engage in longer, more constructive, competitive, and substantively focused but linguistically diverse debates than political moderates. More intense use of Wikipedia policies by politically diverse teams suggests institutional design principles to help unleash the power of politically polarized teams.
1. Introduction
Political diversity may provide fresh perspectives and cognitive resources, but partisan disagreement may also undermine cooperation. The study therefore examines whether politically diverse Wikipedia teams produce higher-quality articles than homogeneous teams.
- Political polarization has increasingly shaped American public discourse and extends beyond politics into consumption of news, cultural products, and scientific information.
- Socially diverse collaborators often combine distinct cognitive resources and perspectives to outperform homogeneous groups on complex or creative tasks.
- Political diversity could improve group production through fresh perspectives or impair it when opponents treat differing information as wrong.
- The study assesses political diversity through approximately four hundred thousand Wikipedia teams working on Politics, Social Issues, and Science articles.
Data and Methods
The study estimates editors’ political alignments from their Wikipedia contributions and relates those alignments to article quality.
- Editors’ political alignments were measured from their relative contributions to conservative versus liberal political articles, then related to article quality using a Wikimedia-developed machine learning algorithm.
Data collection
The dataset is a complete English Wikipedia dump organized into political, social-issues, and science article corpora.
- The dataset contains all edits to English Wikipedia pages from its beginning through 12/01/2016.
- The study focuses on 20,947 Politics pages, 162,085 Social Issues pages, and 49,530 Science pages, approximately 5% of English Wikipedia articles.
- Political pages are divided into Liberal and Conservative sub-corpora constructed from category and subcategory membership.
- Social Issues pages were collected recursively from the Social issues category through four levels of subcategories and include politically salient topics such as homelessness and teenage pregnancy.
Survey of Wikipedia editors
A survey of sampled Wikipedia editors was conducted with community and Wikimedia staff involvement to validate the estimated political-alignment measure.
- Researchers surveyed a random sample of editors with estimated alignment scores while working directly with the Wikipedia community and Wikimedia staff.
Measurement
The paper infers each editor’s political alignment from contributions to liberal versus conservative articles, then uses the spread of those alignments to measure team polarization.
- Measurement: Editors’ political alignment is inferred from the relative bytes they contribute to conservative versus liberal articles.The model treats conservative-article bytes as a binomial variable and estimates each editor’s contribution probability with a Bayesian framework.
- Measurement: Team polarization is measured as the variance of editors’ political alignments across the liberal-conservative spectrum.The variance captures the spread of alignments within a group.
Editors’ political alignments
Wikipedia editors show both politically neutral participation and substantial liberal-conservative polarization, while articles with more editors attract increasingly balanced engagement.
- Editors’ political alignments: Editors’ alignments are measured from their fraction of contributions to conservative versus liberal articles, with Bayesian adjustment for random edits.Editors contributing equally to both sets, or contributing little, are scored near political neutrality.
- Editors’ political alignments: Survey respondents’ party identification correlates at roughly 0.35 with computationally measured conservative-liberal alignment.The survey validates editing history as a noisy behavioral indicator of political preferences.
- Editors’ political alignments: Wikipedia editors have a wide alignment distribution, including a central peak and substantial lower peaks at both political tails.The central peak reflects many minor edits, while the tail peaks indicate editors contributing substantial content to liberal or conservative articles.
- Editors’ political alignments: As the number of editors increases, an article’s average political alignment converges to 0.Articles attracting more attention tend to receive more balanced engagement across the conservative-liberal spectrum.
Effects on Quality
Across Political, Social Issues, and Science articles, greater team political polarization is associated with higher article quality, with the strongest association in Political articles.
- Effects on Quality: Higher team polarization is associated with higher quality in Political, Social Issues, and Science articles.Article quality is measured by a six-category scale from “Stub” to “Featured article,” using a model trained on article content alone.
- Effects on Quality: A 1-unit increase in polarization multiplies the odds of moving to higher-quality categories by 18.57 for Political articles, 2.06 for Social Issues articles, and 1.90 for Science articles.The estimates come from ordinal logistic regression models controlling for article and editor features, including average alignment.
- Effects on Quality: Average editor alignment is negatively associated with quality when editors are biased in either political direction.The regression separates this mean-alignment effect from the positive association between polarization and quality.
Mechanisms of Polarized Collaboration
Politically polarized teams debate fewer topics with more competing framings, sustain more constructive discussion, and invoke Wikipedia policies more often while producing higher-quality articles.
- Mechanisms of Polarized Collaboration: High polarization narrows semantic diversity but increases lexical diversity, producing alternative framings of politically contested subjects.Semantic diversity tracks distinct issues, whereas lexical diversity captures distinct ways of discussing them.
- Mechanisms of Polarized Collaboration: Policy and guideline mentions increase with polarization, especially references to NPOV and OR or NOR.These appeals provide formal standards for governing disagreements over contested knowledge.
- Mechanisms of Polarized Collaboration: Polarized teams debate fewer topics while using more competing terminology and framings.Compared with the least polarized teams, their talk-page discussions contract semantically by 5.6% and expand lexically by 23.4%.
- Mechanisms of Polarized Collaboration: Polarized teams engage in more debate that is less acrimonious.Talk-page analysis treats debate volume and temperature as core aspects of debate intensity.
- Mechanisms of Polarized Collaboration: Editor survey responses describe conflicts over biased content that were often resolved through debate or administrator intervention, sometimes leaving articles in better condition.Reported resolutions included legal arguments, administrator protection, and revisions following intense disagreement.
Discussion
Politically polarized Wikipedia teams can produce higher-quality knowledge, although the observational design limits causal interpretation and collaboration carries social costs. The discussion links this performance to focused debate, information diversity, and policy-governed disagreement, while identifying boundaries for polarization and platform design.
- Findings: Balanced politically polarized teams outperform partisan and moderate groups across Political, Social Issues, and Science articles.The effect is strongest for articles with greater political content, suggesting relevance to the topics considered.
- Mechanisms: Frequent, intense disagreement produces focused debate and higher-quality edits that are more robust and comprehensive.The proposed collaboration mechanisms include debate intensity, information diversity, and use of Wikipedia policies and guidelines.
- Limitations: The observational design prevents a causal conclusion because editors self-select whether to collaborate across political differences.Randomly assigned polarized teams might not outperform homogeneous teams if the observed collaborators differ systematically from those who avoid such collaborations.
- Costs and governance: Politically diverse collaboration can feel worse to participants, but Wikipedia policies and norms help regulate ideological conflict.Balanced competition may soften conflict by enabling members to police tone and content, with moderators available when norms break down.
- Boundary conditions: Quality may eventually decline at extreme polarization, but the estimated optimum exceeds the polarization reached by 95% of teams.Among the 5% most polarized teams, no statistically significant relationship between polarization and quality was found.
- Implications: The findings suggest that crowd-sourcing platforms may benefit from motivated contributors with partisan perspectives rather than discouraging all user bias.The paper proposes seeking balanced, diverse perspectives and enforceable policies when constructing environments for contested knowledge.
Appendix C. Survey measure of political alignment
The survey assessed whether computational political-alignment scores corresponded to editors’ self-reported political identification. Across comparisons, the measure showed moderate validity, while response rates varied across alignment ranges.
- Survey sample: 54% of respondents (64/118) lived outside the United States, and ambiguous party-identification responses were excluded.
- Survey response: 24% (118/500) of solicited editors responded, with responses received from every alignment quintile.Only 1 of 27 solicited editors in the [0.2, 0.6) alignment range replied.
- Measure validation: The strictest comparison found a 0.31 correlation between computational alignment and self-reported identification among 21 U.S.-based responses.This comparison used raw survey responses without recoding.
- Measure validation: Including recoded responses and non-U.S. respondents produced a statistically significant correlation at n=47 (p=0.02), within the range of earlier comparisons.
- Measure validation: Across all three comparisons, computational and self-reported political alignment correlated moderately at 0.31-0.35, indicating a noisy but valid measure.The authors focus on the larger U.S.-based comparison that avoids subjective recoding of non-U.S. parties.
- Correlational analysis: Polarization was positively correlated with quality, lexical diversity, activity, and policy mentions, but negatively correlated with semantic diversity and talk-page temperature.These associations were interpreted as consistent with greater debate volume, competing framings, and institutional engagement.
Appendix D.2. Regression Analysis
The regression analysis estimated how polarization related to talk-page deliberation while controlling for relevant page features and accounting for correlated predictors. A structural equation model then combined proposed pathways from polarization to article quality.
- Regression models: Multiple linear regressions tested polarization’s associations with debate volume, information diversity, policy mentions, and talk-page temperature.The models controlled for relevant Wikipedia talk-page and article-page features.
- Regression models: All models controlled for the number of talk-page editors, while page length and article diversity were included where appropriate.
- Model specification: Substantial collinearity prevented including every variable in every model, motivating simpler models and composite factors.The authors illustrate this with correlations among editor counts, edit counts, and page length.
- Structural equation model: The structural equation model represented debate volume, institution, and article activity as latent variables built from observed editing and discussion measures.The model was estimated using the lavaan R package.
- Structural equation model: 0.624 was the combined estimated effect of polarization on quality through debate volume, lexical diversity, semantic diversity, institutional use, and talk temperature.Individual path effects ranged from 0.035 through institution to 0.230 through semantic diversity.
- Model evaluation: All estimated structural-model parameters were significant at p < 0.001, but fit indexes were modest: CFI=0.78 and NNFI=0.71.The authors report that adding an article-activity-on-volume path raised CFI to 0.89 but reduced interpretability.