Source-linked AI summary
Using a Model of Social Dynamics to Predict Popularity of News
Kristina Lerman, Tad Hogg
TL;DR
Social-media popularity is highly unequal and difficult to predict because quality, visibility, and social influence interact. The paper uses a stochastic model of Digg’s user behavior and interface to infer story interest from early votes and predict longer-term popularity. The model outperforms direct extrapolation and predicts promotion accurately, while finer-grained modeling of audience differences remains an avenue for improvement.
Problem
Popularity is uneven and difficult to predict because content quality, site visibility, and social influence interact, despite prediction’s importance for platforms and users.
Method
A stochastic model of Digg’s user dynamics and interface estimates a new story’s interest from early voting and predicts its eventual votes or promotion.
Results
The model correctly predicts promotion for 95% of stories and achieves 87% correlation with observed votes, versus 75% for early-rate extrapolation.
Takeaways & Limitations
Modeling social dynamics separates story interest from interface effects and enables early prediction of popularity from observed voting dynamics.
Abstract
from arXiv · showhide
Popularity of content in social media is unequally distributed, with some items receiving a disproportionate share of attention from users. Predicting which newly-submitted items will become popular is critically important for both companies that host social media sites and their users. Accurate and timely prediction would enable the companies to maximize revenue through differential pricing for access to content or ad placement. Prediction would also give consumers an important tool for filtering the ever-growing amount of content. Predicting popularity of content in social media, however, is challenging due to the complex interactions among content quality, how the social media site chooses to highlight content, and influence among users. While these factors make it difficult to predict popularity \emph{a priori}, we show that stochastic models of user behavior on these sites allows predicting popularity based on early user reactions to new content. By incorporating aspects of the web site design, such models improve on predictions based on simply extrapolating from the early votes. We validate this claim on the social news portal Digg using a previously-developed model of social voting based on the Digg user interface.
I. INTRODUCTION
Social-media popularity is highly unequal and difficult to predict because content quality, user responses, site design, and social influence interact. The paper proposes using a stochastic model of Digg’s user dynamics and interface to predict popularity from early reactions.
- Only a handful of Digg’s more than 16,000 daily submissions become wildly popular, while most receive at most the submitter’s vote.
- Early and late popularity are somewhat correlated, but the factors driving success remain unclear.
- Social influence contributes to inequality and unpredictability of success, while content quality contributes only weakly to eventual success.
- The study uses Digg data to test whether modeling collective user behavior can predict a story’s popularity from users’ early reactions.
- Digg promotes about a hundred stories daily from thousands of submissions, using a proprietary algorithm that predicts which stories will attract many votes.
A. User interface
Digg’s interface controls story visibility through upcoming-story, front-page, and friends-activity views. These design features determine how users encounter stories and how votes expose them to additional users.
- A. User interface: New stories appear chronologically in the upcoming list, while promoted stories appear chronologically on the front page, with 15 stories per page.
- A. User interface: The figures document Digg’s front-page layout and the evolution and distribution of votes for front-page stories.
- A. User interface: A newly submitted story is visible in the upcoming list and to the submitter’s fans through the friends interface.
- A. User interface: Each vote makes a story visible through the friends interface to the voter’s fans, who can then discover and vote on it.
- A. User interface: Digg also offers views of previously popular stories and recommendations based on users with similar voting histories, though recommendations were unavailable during data collection.
B. Inequality of popularity
Digg stories exhibit strongly unequal popularity: a few promoted stories attract thousands of votes, while most receive only a few hundred or less. The paper links this pattern to visibility, social influence, and story interest.
- B. Inequality of popularity: After promotion, stories accumulate votes rapidly; as they age, vote accumulation slows and eventually saturates at a final vote count.
- B. Inequality of popularity: A handful of front-page stories receive thousands of votes, while most receive only a few hundred; never-promoted stories often receive a single vote.
- B. Inequality of popularity: Long-tailed popularity distributions recur across cultural artifacts and online behaviors, although their exact shapes differ among social-media sites.
- B. Inequality of popularity: Modeling social influence through Digg’s friends interface is intended to make the popularity created by that influence quantitatively predictable.
- B. Inequality of popularity: The Digg voting model represents users as stochastic processes whose states correspond to actions such as following links, voting, and befriending.
A. Data sets
The study combines Digg story and social-network data with a stochastic model that represents voting as changing aggregate visibility and interest. The model incorporates story position, page navigation, promotion, friends-interface exposure, and story-specific parameters.
- Data sets: The May data set contains 2,152 Technology stories submitted by 1,212 users, including 510 promoted stories from 239 users.Stories were periodically scraped, with at least four vote observations per story.
- Data sets: The June data set includes 201 promoted stories and 159 additional stories receiving at least 10 votes.The promoted stories include the first 216 voters for each story.
- Aggregate story state: The model tracks a story’s vote count, list membership, and position within the upcoming or front-page list.These aggregate state variables describe how a story changes after submission.
- Aggregate story state: Voting depends on story interest and visibility through front-page, upcoming-page, and friends interfaces, with user behavior represented stochastically.The rate terms represent users finding stories through these interfaces and voting on them.
- Interface visibility: Digg lists contain 15 stories per page, and the model represents decreasing visibility with fractional list positions that advance as stories are submitted or promoted.For example, p = 1.5 denotes a story halfway down the first page.
- Promotion and rates: Promotion is modeled by moving a story from upcoming pages to the front page when its accumulated votes exceed threshold h.The upcoming-list exposure is also limited to 24 hours.
- Model parameters: The model uses story interestingness r and submitter-fan count S as story-specific parameters, while other parameters are fixed or calibrated from the data.The initial friends-interface state equals S, and the friends-growth parameters include a = 51 and b = 0.62.
C. Model parameters and solutions
The model’s solutions depend on story interestingness and submitter connectivity, and reproduce observed Digg voting and promotion patterns. It predicts promotion accurately while separating the roles of story quality and social influence.
- Model parameters and solutions: The model fits observed story-vote evolution while varying only interestingness r and submitter-fan count S across stories.The remaining model parameters are fixed in Table I.
- Promotion prediction: 95% of stories in the May data set are classified correctly as promoted or not promoted by the model.The promotion boundary is expressed as a curve in S–r space.
- Promotion prediction: Among promoted stories, the correlation between S and r is −0.13, significantly different from zero with p-value less than 10^-4.Poorly connected submitters tend to need higher story interest for promotion.
- Observed dynamics: Higher-r stories are promoted faster and receive more votes, while higher-S users can promote lower-r stories more successfully.The model also reproduces slow initial growth before promotion.
- Interpretation: Estimating r from voting dynamics separates story quality from social influence and supports quantitative analysis of their effects on Digg popularity.The approach uses observed early voting together with information about user connections.
IV. MODEL-BASED PREDICTION
The paper presents social-dynamics modeling as a way to predict popularity from users’ early reactions. On Digg, the model also separates story quality from social influence.
- Model-based prediction: A social-dynamics model uses early user reactions to predict a newly submitted item’s eventual popularity.The paper investigates this application with real-world Digg data.
- Model-based prediction: The model separates the effects of story quality and social influence on popularity.This separation supports both quality estimation and eventual-popularity prediction.
A. Estimating story quality
The model estimates story interestingness from observed voting dynamics and finds broad variation in r among promoted stories. The estimated distribution is consistent with a lognormal form.
- Estimating story quality: Story interestingness r is estimated by minimizing RMS difference between observed and model-predicted votes.The fit uses data through the sample end or two days after submission, whichever comes first.
- Estimating story quality: 14% is the RMS relative error between observed and predicted votes for 510 promoted May stories.The estimate is based on model fits to the promoted-story data.
- Distribution of interestingness: The 510 promoted stories span a wide range of estimated interestingness values, with r fitting a lognormal distribution.The estimated mean and standard deviation of log(r) are −1.67 ± 0.04 and 0.47 ± 0.03, respectively, with 95% confidence intervals.
- Examples: The selected table compares stories with the highest and lowest r values using final votes, estimated r, and story title.It illustrates how estimated interestingness is reported alongside popularity outcomes.
- Distribution of interestingness: The June r values show a similar lognormal distribution, allowing the model to factor interface effects out of overall interestingness variation.The resulting measure reflects users’ inclination to vote when they see a story.
B. Predicting final number of votes
The model estimates a story’s interestingness from early votes and uses it to predict eventual vote totals. Early observations strongly predict relative popularity, though prediction quality varies across data sets.
- 34% relative error and 87% correlation result when estimating May stories’ final votes from their first four observations.These early observations provide a strong prediction of the relative ordering of stories by eventual votes.
- Early voting reactions can predict eventual ratings or popularity on Digg and YouTube.
- The model predicts June stories less accurately, with 0.49 correlation, RMS error 593, and 23% of variance explained.The estimates use each story’s first 20 votes and model parameters determined from the May data set.
C. Comparing to direct extrapolation
The social-dynamics model separates interface visibility from underlying user interest and improves prediction over direct extrapolation from early vote rates. Its accuracy differs across data sets and prediction settings.
- 75% correlation from direct extrapolation is lower than the model’s May-data prediction, with the difference statistically significant at p-value less than 5 × 10^-4.The direct-extrapolation method also has a larger RMS error and does not model fan growth.
- Modeling average fan growth improves the description of vote accumulation by accounting for changing story visibility.The model separates interface visibility from the underlying rate at which users vote when they see a story.
- The May model uses early observations to estimate r, while Figures 5 and 6 compare predicted and observed final votes for May and June promoted stories.Figure 5 uses the first four observations; Figure 6 uses the first 20 votes.
- 0.44 correlation, RMS error 610, and 19% variance explained show model-based prediction improves over direct extrapolation for June stories.
D. Comparing to social influence only prediction
The paper compares its model with a social-influence-based classifier that uses early fan-vote patterns to predict whether stories exceed 505 votes. Social influence is measured through visibility via the friends interface.
- Stories receiving many initial fan votes ultimately accumulate fewer votes than stories receiving few fan votes.The relationship is shown for 201 front-page stories in the June data set, with median final votes and outliers removed.
- Social influence is measured as the proportion of initial votes coming through fans of the submitter or previous voters.The paper assumes discovery through the friends interface indicates social influence.
- The classifier uses three attributes: early fan votes, submitter fan count, and whether the story exceeds 505 votes.
- The first 10 votes can provide an early basis for the social-influence prediction.
- The classifier predicts whether upcoming stories will exceed 505 votes, particularly for stories submitted by top users with larger, more active fan networks.
V. DISCUSSION
The discussion identifies assumptions and data differences that reduce prediction accuracy, while outlining refinements to promotion and social-influence modeling. These limitations motivate uncertainty bounds and more detailed estimates of exposure and interest.
- Data and calibration: June predictions were worse partly because the first 20 votes covered shorter periods than May’s fourth observations.May observations often occurred about four hours after submission, while June’s first 20 votes generally represented shorter periods.
- Data and calibration: June predictions also used parameters calibrated on May data, and recalibration could improve performance but was not tested.The authors could not explore June-specific calibration because they lacked relevant data.
- Promotion modeling: Treating promotion as an exact 40-vote threshold is inaccurate because some stories were promoted earlier; prediction can instead report bounds reflecting promotion uncertainty.The model assumes a threshold, but June data showed promotion before stories reached 40 votes.
- Social influence: Using expected growth in new fans is a simplification; computing exposure from each voter’s network could better model highly connected users.The proposed refinement targets cases where users such as kevinrose expose stories to many people.
- Social influence: A finer-grained social-influence model could improve estimates of interest and final votes by distinguishing fans from non-fans.The authors also suggest monitoring interest spread through fan networks to estimate r more accurately.
VI. RELATED WORK
Related work links popularity to social dynamics, early reactions, and content quality, but differs in aggregation level and mechanism. This paper models microscopic user behavior on Digg and reports stronger prediction than social-influence-only or early-to-late correlation methods.
- Social dynamics: Social Web data supports research on public-opinion trends and information flow in groups.The cited work uses behavioral data from large groups to study detection, influence, and information dynamics.
- Popularity and quality: Popularity and user activity exhibit broad distributions across social media sites, and vote trajectories can distinguish high-quality videos from junk.Prior work examined both general popularity distributions and time-varying video votes.
- Modeling level: Unlike aggregate studies of many individuals, this method models microscopic dynamics showing how individual behavior contributes to content popularity.The comparison distinguishes the paper’s individual-level mechanism from prior large-scale statistical aggregation.
- Early prediction: Early and late popularity are significantly correlated on Slashdot, Digg, and YouTube, enabling long-term prediction from early observations.Szabo and Huberman predicted Digg popularity after 30 days using popularity one hour after front-page promotion.
- Research gap: Popularity is weakly related to inherent quality, while social influence contributes to unequal and difficult-to-predict outcomes.This motivates modeling social dynamics rather than relying only on content quality or simple popularity extrapolation.
- Paper’s contribution: Model-based prediction outperforms social-influence-only and early-to-late-vote methods, with further gains possible from distinguishing fans’ and general users’ interest.The model estimates story interest from early vote evolution before predicting votes over several days.