Source-linked AI summary
Quantifying social organization and political polarization in online platforms
Isaac Waller, Ashton Anderson
TL;DR
Online communities are difficult to characterize because digital discussion is pseudonymous, unstructured, and large-scale. The paper uses neural embeddings and complete Reddit activity to quantify social dimensions and polarization, finding a major 2016 platform-level shift driven mainly by new users and right-wing activity rather than widespread individual polarization.
Problem
Pseudonymous, unstructured, large-scale online discussion limits measurement of communities’ social makeup and the dynamics and mechanisms of polarization.
Method
The paper embeds communities using aggregate behavioural patterns and analyzes complete longitudinal Reddit activity to measure social dimensions and platform- and individual-level polarization.
Results
Reddit polarization rose sharply around the 2016 election and remained elevated, with individual polarization rare and the increase disproportionately driven by new and newly political users.
Takeaways & Limitations
The methodology can quantify online social organization and situate content and behaviours in their platforms’ social contexts.
Takeaways & Limitations
The partisan dimension more closely captures Republicanism in general, underscoring the importance of validating community scores.
Abstract
from arXiv · showhide
Optimism about the Internet's potential to bring the world together has been tempered by concerns about its role in inflaming the 'culture wars'. Via mass selection into like-minded groups, online society may be becoming more fragmented and polarized, particularly with respect to partisan differences. However, our ability to measure the social makeup of online communities, and in turn understand the social organization of online platforms, is limited by the pseudonymous, unstructured, and large-scale nature of digital discussion. We develop a neural embedding methodology to quantify the positioning of online communities along social dimensions by leveraging large-scale patterns of aggregate behaviour. Applying our methodology to 5.1B Reddit comments made in 10K communities over 14 years, we measure how the macroscale community structure is organized with respect to age, gender, and U.S. political partisanship. Examining political content, we find Reddit underwent a significant polarization event around the 2016 U.S. presidential election, and remained highly polarized for years afterward. Contrary to conventional wisdom, however, individual-level polarization is rare; the system-level shift in 2016 was disproportionately driven by the arrival of new and newly political users. Political polarization on Reddit is unrelated to previous activity on the platform, and is instead temporally aligned with external events. We also observe a stark ideological asymmetry, with the sharp increase in 2016 being entirely attributable to changes in right-wing activity. Our methodology is broadly applicable to the study of online interaction, and our findings have implications for the design of online platforms, understanding the social contexts of online behaviour, and quantifying the dynamics and mechanisms of online polarization.
Social dimensions in community embeddings
The embedding methodology reveals that Reddit communities are organized along age, gender, and partisan dimensions, with substantial structure both across and within behavioural clusters. Political activity is selectively concentrated in ideologically homogeneous communities, and polarization rose sharply around the 2016 election, driven especially by right-wing activity and newer users.
- Macroscale organization: Behavioural clusters are socially stratified: top-level groups depart from a uniform score-percentile distribution, showing that age, gender, and U.S. partisanship organize Reddit activity.Because clustering uses behavioural relationships, this stratification indicates that clusters differ socially rather than only topically.
- Macroscale organization: Politics communities are polarized along the partisan axis, with 77% below the 20th percentile or above the 80th percentile, while every cluster contains communities on both sides of each global mean.This combines strong cluster-level polarization with substantial inter- and intra-cluster diversity.
- Cross-dimensional structure: Partisan, gender, and age dimensions are related but differ from offline patterns: masculine-leaning communities skew right-wing (r=-0.29), while older communities skew left-wing and younger communities skew right-wing (r=-0.37).At the extremes, 44.0% of the most left-wing communities are feminine-leaning, whereas 23.3% of the most right-wing communities are masculine-leaning.
- Political polarization: Political activity is selectively concentrated in ideologically homogeneous communities, with users contributing disproportionately to communities matching their partisan orientation.Only 8% of discussion occurs in the most left-wing communities, but left-contributing users place an average of 44% of their activity there; corresponding right-wing figures are 16% and 62%.
- Political polarization: Reddit became substantially more polarized around the 2016 presidential election, peaking at 1.86 in November 2016 after remaining between 1.08 and 1.28 through 2015.Far-left and far-right activity rose from 2.8% in January 2015 to 24.8% in November 2016, and polarization remained elevated afterward.
- Political polarization: Individual-level polarization was uncommon: the fraction of users shifting at least one standard deviation toward a partisan pole was 1.9–3.3% before 2016 and peaked at 11.3% in November 2016.Within-cohort polarization usually remained unchanged or decreased, except during 2016.
Data
The study analyzes Reddit’s complete public commenting history to characterize multi-community participation and create community embeddings from behavioral similarity.
- 5.1B comments from 2005 through 2018 form the study’s complete Reddit dataset.The archive covers comments made since Reddit comments were introduced through the end of 2018.
- 34.7M Reddit commenters are represented with their complete public commenting histories, subreddit memberships, and comment timestamps.
- 52.9% of users commented in more than one subreddit, with a mean of 9.6 subreddits per user.This multi-community activity provides information about behavioral similarity between subreddits.
- Behavioral similarity across subreddits is harnessed to create community embeddings and social dimensions.
Creating the community embedding
The paper represents Reddit communities from user–community commenting interactions using SGNS embeddings, where vector similarity captures behavioral overlap and higher-order community relationships.
- Each user–community comment instance becomes a word-context pair, with repeated comments producing repeated training examples.
- 10,006 subreddits account for 95.4% of comments and 93.2% of users in the training representation.
- 72% of 4,392 community analogies are solved perfectly, while 96% are solved nearly perfectly with the correct answer in the top five.
- SGNS learns community and user vectors by increasing dot products for frequent co-occurrences and decreasing them for random negative examples.
- SGNS captures higher-order context overlap that PMI does not, although the exact relationship between cosine similarity and user overlap remains unresolved.
- Agglomerative clustering partitions communities into 30 clusters, which are manually labeled by dominant topic.
Finding social dimensions
The authors construct social dimensions by contrasting similar community pairs that differ in a target social construct, then aggregate those contrasts into robust embedding directions.
- The method identifies social dimensions in community embeddings to quantify the social makeup of online communities.
- A seed pair should contain communities that are extremely similar except for the target social construct.
- The algorithm augments each seed with nearest-neighbor pairs whose vector differences resemble the seed-pair difference.
- Greedy selection chooses non-overlapping pairs until k pairs define the dimension, after which their vector differences are averaged.
- The analysis uses k=10, selecting nine additional pairs; fewer pairs were less robust, whereas more produced extremely similar axes.
- Dimensions cover age, gender, partisanship, affluence, time, sociality, and edginess, and can extend beyond one-dimensional identity representations.
Computing community scores
Community scores are obtained by projecting normalized community vectors onto dimension vectors, with scores reflecting relative similarity to the paired communities.
- Each community receives a score by projecting its normalized vector onto the corresponding social-dimension vector.
- A score is proportional to average similarity with one side of the dimension minus average similarity with the other side.
- Communities closer to one seed occupy the dimension’s poles, while communities equidistant from both seeds receive a score of 0.
- The authors compute scores for all 10,006 communities across age, gender, partisan, and affluence dimensions.
- Scores are robust to small seed changes: age, gender, and partisan dimensions correlate with their B versions at r=0.90, r=0.86, and r=0.55, respectively.
- The partisan B dimension differs because Trump-supporter behavior captures a more specific online construct than general Republicanism.
Validating community scores
The social scores are validated against manually labeled communities, occupational demographics, university–city contrasts, Census income, and presidential-election voting patterns.
- Occupation gender composition correlates with community gender scores at r=0.89, including occupations at the extremes and in the middle.
- University communities are strongly differentiated from city communities on age, with r=0.91 and Cohen’s d=4.37.
- The manual political labels strongly correlate with partisan scores at r=0.92, with Cohen’s d=4.89.
- Across metropolitan areas, median household income correlates with affluence scores at r=0.39, while the 2016 Republican–Democrat vote differential correlates with partisan scores at r=0.42.
- The partisan correlation indicates that the method captures online partisanship, which differs from its traditional offline analogue.
Measuring relationships between dimensions
On Reddit, age, gender, and partisan community scores show weak-to-moderate relationships, with partisan position more associated with age and gender than age is with gender.
- Age and gender are weakly correlated (r= 0.10).
- Gender and partisan scores have a moderate negative correlation (r= −0.29).
- Age and partisan scores have a moderate negative correlation (r= −0.37).
- Alternate B-axis analyses produce similar relationships across partisan, gender, and age dimensions.Reported correlations include partisan B–gender (r= −0.34), partisan B–age (r= −0.13), partisan–gender B (r= −0.26), and partisan–age B (r= −0.33).
Computing word scores
Word scores summarize the communities in which words are used, while capping repeated contributions from individual commenters to reduce bot-related distortion.
- Word scores are community-score averages weighted by each word’s usage across communities.They represent the types of communities in which a word is likely to be observed.
- Usage by one commenter in a subreddit is capped at 100 counted instances per word.This cap is intended to avoid distortion from bots repeatedly reusing the same words.
- The most extreme word scores on each primary axis are reported in Appendix Figure 1.
Measuring political polarization
The analysis identifies political communities and quantifies partisan positioning using embedding-derived scores, while testing whether selective activity reflects genuine self-selection rather than methodological artifacts.
- 553 communities, or 5.53%, are labeled political, and the classification correctly categorizes 92% of manually coded political communities.
- A community’s partisan z-score is standardized against the full partisan-score distribution, with negative values indicating left-wing association and positive values right-wing association.A z-score of 0 denotes the overall mean.
- Deleted and non-deleted comments have extremely similar partisan-score distributions, with mean difference −0.01 and D_KL(P∥Q) = 0.033 bits.The authors therefore conclude that excluding deleted comments is reasonable, while acknowledging that deleted comments could differ enough to affect findings.
- Political activity is grouped into five partisan bins spanning left wing, leaning left, center, leaning right, and right wing.The bins use z-score thresholds below −2, −2 to −1, −1 to 1, 1 to 2, and above 2.
- The self-selection analysis measures how often authors contributing in one partisan bin also contribute in another bin, weighting authors by their comment counts.Same-bin activity estimates the average concentration of an author’s activity within that bin.
- In a randomly shuffled-author dataset, bin distributions differed from the overall activity distribution by less than 0.85% in same-bin activity.This supports using the overall distribution as a reference for the no-self-selection condition.
Measuring dynamics of polarization
Polarization dynamics are measured at both platform and individual levels by tracking partisan activity over time, user cohorts, and the contributions of new versus existing users.
- Platform-level dynamics: Platform-level polarization is tracked through monthly distributions of partisan activity and the share occurring in very left- or right-wing communities.Very left-wing communities have z< −3, while very right-wing communities have z> 3.
- Individual-level dynamics: Individual polarization is estimated using users’ average absolute partisan z-scores across activity and over time.Absolute scores capture polarization regardless of whether activity is far left or far right.
- Individual-level dynamics: Users are divided into ten cohorts by the year of their first political-community comment, and cohort polarization is tracked over time.Additional comparisons use account age and the number of active political months.
- Individual-level dynamics: Within-user polarization is assessed by comparing average partisan scores across calendar months, requiring at least 10 comments per user-month.The analysis also reports correlations between users’ average partisan scores in month pairs.
- New and existing users: New users are defined as commenters without political-community activity in the prior 12 months, and their contribution is separated from existing users’ contribution.The decomposition accounts for both activity shares and average polarization levels.
- New and existing users: The new-user and existing-user components sum exactly to the overall year-over-year change in platform polarization.This decomposition avoids comparing new users at one time with users who later become existing users.
- Ideological asymmetry and implicit activity: Left-wing and right-wing activity are analyzed separately, alongside tests of links between implicit and explicit partisan activity.Implicitly partisan communities are low on partisan-ness but at least two standard deviations from the global partisan mean.
age
The age dimension organizes Reddit communities from young to old, with distributions examined across clusters and validated against community labels. Age is strongly related to the distinction between university and city communities.
- age: Age scores organize communities along a young–old dimension.The plotted axis ranges from young to old.
- age: Age-score distributions are reported separately for community clusters, with outliers annotated beyond two standard deviations.Dashed lines indicate the global mean on the age dimension.
- age: r=0.91 and Cohen’s d=4.37 show a strong relationship between age scores and university-versus-city community labels.The validation also describes a gap between university and city communities on the age dimension.
Supplementary Information
The supplementary information documents referenced communities and the labeling of their social-dimension scores. Community descriptions were collected in 2019 and 2020, with some descriptions missing because communities had closed or been banned before scraping.
- Supplementary Information: Supplementary Table 1 provides a glossary of referenced communities.The glossary lists communities alongside their descriptions.
- Supplementary Information: Some community descriptions are missing because descriptions were scraped in 2019 and 2020.Communities that had closed or been banned before scraping are included among those with missing descriptions.
- Supplementary Information: Each community is labeled with scores on the main dimensions by distance from the mean.The labeling covers the paper’s main social dimensions.