Source-linked AI summary
Governance in Social Media: A case study of the Wikipedia promotion process
Jure Leskovec, Daniel Huttenlocher, Jon Kleinberg
TL;DR
The paper addresses the limited study of deliberation in social-media governance by analyzing Wikipedia’s extensive, public, recorded adminship promotion elections. It examines voter–candidate relationships and sequential voting, finding relative assessment, heterogeneous voter responses, and strong dependence on early votes alongside limited general ordering effects.
Problem
Deliberation is important in social-media governance, but its decision-making processes have been relatively little investigated because detailed records are difficult to find.
Method
The paper analyzes Wikipedia’s public, recorded adminship promotion process from the perspective of voters and election dynamics.
Results
Voter support depends strongly on voter–candidate characteristics, frequent voters have diverse response functions, and election success depends heavily on early votes while most initial-vote ordering effects are weak.
Takeaways & Limitations
Social-media group decisions exhibit relative assessment and heterogeneous individual behavior, while aggregate sequential dynamics need not reflect a typical voter or strong herding.
Takeaways & Limitations
The paper identifies integrating temporal analysis with Bayesian cascade models and textual analysis of deliberation as important directions for further work.
Abstract
from arXiv · showhide
Social media sites are often guided by a core group of committed users engaged in various forms of governance. A crucial aspect of this type of governance is deliberation, in which such a group reaches decisions on issues of importance to the site. Despite its crucial --- though subtle --- role in how a number of prominent social media sites function, there has been relatively little investigation of the deliberative aspects of social media governance. Here we explore this issue, investigating a particular deliberative process that is extensive, public, and recorded: the promotion of Wikipedia admins, which is determined by elections that engage committed members of the Wikipedia community. We find that the group decision-making at the heart of this process exhibits several fundamental forms of relative assessment. First we observe that the chance that a voter will support a candidate is strongly dependent on the relationship between characteristics of the voter and the candidate. Second we investigate how both individual voter decisions and overall election outcomes can be based on models that take into account the sequential, public nature of the voting.
Introduction
The paper studies deliberation in social-media governance through Wikipedia’s public, recorded adminship promotion process. It examines how voter–candidate relationships and sequential voting shape individual decisions and election outcomes.
- Introduction: Social-media governance includes deliberation by core participants, but the deliberative process has been comparatively difficult to study because decision records are often unavailable.Wikipedia provides detailed public traces of group decision-making by a committed community.
- Introduction: Wikipedia adminship elections provide a formal, public setting in which any user may vote after discussion and deliberation, with bureaucrats interpreting the results.Candidates submit a promotion case, followed by community discussion and voting; the process is recorded in Wikipedia archives.
- Introduction: The probability that a voter supports a candidate depends strongly on relative voter–candidate characteristics, including edits, barnstars, and prior interaction.Positive voting is modeled as a function of both the candidate and the voter rather than the candidate alone.
- Introduction: Frequent voters show strikingly diverse response functions: some are stingy with positive votes, while others are much freer.Aggregate response curves may therefore average across highly heterogeneous individual behaviors rather than represent a typical voter.
- Introduction: Election success depends heavily on the first few votes, while the interleaving order of initial positive and negative votes generally has little effect except in structured cases.The finding contrasts with stronger predictions of herding behavior and information-cascade models.
Related Work
Related work examines governance enforcement and sequential online opinion processes, while prior Wikipedia research modeled promotion from candidate characteristics rather than voter deliberation.
- Related Work: Prior social-media governance research has addressed enforcement, including how norms and rules can control deviant behavior in online communities.Enforcement is distinct from this paper’s focus on deliberation.
- Related Work: Studies of Amazon reviews and Digg votes investigate how online opinions are expressed and evaluated sequentially by user communities.These works provide related examples of public, sequential deliberation in online settings.
- Related Work: Burke and Kraut modeled Wikipedia promotion outcomes using textual features of candidates’ contributions and user activity.Their analysis focused on candidate characteristics rather than voters.
Dataset description
The dataset contains publicly recorded English-Wikipedia adminship elections and signed votes, with support, oppose, and neutral categories. The analyses restrict attention to sufficiently large elections and use each user’s last recast vote.
- Dataset description: 2,794 English-Wikipedia adminship elections were collected between September 17, 2004 and January 6, 2008.Each election includes a public discussion and vote, with signed votes retained in Wikipedia archives.
- Dataset description: 114,040 votes comprise 83,962 support, 23,118 oppose, and 6,960 neutral votes, yielding a baseline support probability of 0.784.Support votes were discussed 7% of the time, compared with 82% of oppose votes.
- Dataset description: 44.6% of elections resulted in successful promotion, and successful elections averaged 52.2 support, 1.5 neutral, and 3.1 oppose votes.Successful elections averaged 94.7% support among all votes.
- Dataset description: 8,298 distinct users participated as voters and/or candidates, including 7,499 voters and 2,539 candidates.Candidates can appear in multiple elections before being elected.
- Dataset description: The study discards neutral votes and analyzes only elections with at least 10 votes, retaining the last vote when a user recasts it.These preprocessing choices define the analysis sample rather than the full collection of votes.
Relative Merit of Candidates and Voters
Wikipedia voters assess candidates partly through relative merit: support depends on how a candidate’s edits and barnstars compare with the voter’s, with especially low support near comparable achievement levels. Direct communication between voters and candidates is also associated with greater support.
- Relative assessment: Support probability depends on relative differences between candidate and voter merit, rather than only on candidate attributes.The analysis examines differences in edits and barnstars as univariate measures of relative merit.
- Relative merit measures: Figure 3 compares support probability across differences in edits and barnstars, with both measures showing a similar non-monotonic pattern around zero.The curves drop as differences move from negative to positive and then rebound around larger positive differences.
- Edit-count comparisons: Voters are least likely to support candidates whose edit counts are approximately equal to their own.Support rebounds once the signed log-difference exceeds 1, but remains below baseline when voters have more edits.
- Interpretation: The shared non-monotonicity in edit and barnstar comparisons suggests voters are particularly critical of candidates with achievement levels comparable to their own.This contrasts with a purely monotonic relationship between relative achievement and support.
- Direct interaction: The probability of a positive vote tends to increase with the amount of direct communication between voter and candidate.The analysis measures communication through talk-page edits and exchanged words.
Thresholds and Diversity in Voter Behavior
Wikipedia voters respond to both the current balance of public votes and their own histories and identities. Aggregate response patterns conceal substantial individual diversity, subgroup differences, and temporal change.
- Sequential voting: Voters’ positive-vote probability is evaluated against the fraction of preceding positive votes, using f(x) to capture responses to public voting precedent.The diagonal f(x) ≈ x is the coin-flip baseline tied to the current positive-vote fraction.
- Sequential voting: f(x) exceeds x below about 0.3 and falls below x above 0.3, especially above 0.6, indicating fewer positive votes when support is already prevalent.The pattern is consistent with greater willingness to express an opinion against the prevailing outcome.
- Subgroup differences: Non-admins are more generous than admins when the preceding positive-vote fraction is low, while the two groups agree closely at high values of x.For admins, f(x) ≈ x holds more closely when x is small.
- Individual diversity: Individual response functions vary considerably: some frequent voters are overwhelmingly negative below 70% prior support, whereas others vote positively despite mostly negative preceding votes.The estimates cover 11 users with more than 400 votes and 28 users with more than 300 votes.
- Individual diversity: Population-level response functions can average highly diverse individual functions, so aggregate behavior may not represent a typical voter.The paper highlights this as an issue for interpreting population aggregates in social data.
- Temporal change: Over time, voters generally become more conservative with positive votes, particularly non-admins, while frequently voting admins show more stable aggregate behavior.The comparison uses first- versus second-half election histories for 78 users who voted in more than 200 elections.
Dynamics of Elections over Time
The paper examines how the sequence and public visibility of votes shape Wikipedia promotion elections. Vote order usually has little effect on final outcomes, but early voting patterns reveal systematic differences between successful and unsuccessful elections.
- Election histories: Unsuccessful elections tend to begin with disproportionately positive votes and decline over time, whereas successful elections maintain a relatively stable positive-vote fraction.The decline in unsuccessful elections reaches the randomized baseline, while successful elections resemble randomly permuted vote orders.
- Early prefixes: For length-3 prefixes with a 2:1 tally, (support,support,oppose) and (oppose,support,support) had success rates of 0.328 and 0.323, while (support,oppose,support) had 0.277.The prefix beginning with support followed by oppose was associated with the lower success rate.
- Early prefixes: The ordering contrast differs from standard herding expectations because early votes by a candidate’s endorsers create selection effects.In this setting, initial positive votes do not uniformly elevate the probability of success.
- Timing effects: The position of one oppose vote had essentially no effect when the first 8 votes were 7 support and 1 oppose, or the first 15 were 14 support and 1 oppose.Timing effects were therefore more noticeable in unsuccessful elections than in strongly positive ones.
Conclusion
The study analyzes Wikipedia promotion elections as a case of social-media governance and identifies relative assessment in voter decisions and election dynamics. It also points toward combining temporal, network, and textual analyses to study deliberation further.
- Findings: The study finds that voter decisions reflect relative characteristics of voters and candidates, as well as the election’s state when votes are cast.These are presented as forms of relative assessment in group decision-making.
- Findings: The analysis identifies ordering effects in elections that contrast with standard theories of herding and information cascades.The conclusion frames these effects as part of the study of social-media governance.
- Future directions: Future work could connect relative merit to social-network comparison, integrate election dynamics with Bayesian cascade models, and combine structural analysis with discussion text.These directions are proposed as ways to pursue further questions about governance and deliberation.