Expecting to be HIP: Hawkes Intensity Processes for Social Media Popularity

• The work addresses open questions about popularity under continuous external influence, whether promotion can make content viral, and forecasting popularity given planned promotions.Prior approaches described popularity using fixed stylistic prototypes, while popularity-history methods produced competitive estimates but did not address forecasts given planned promotions.Source: §1 · p. 1
• The study links popularity for 81.9 million YouTube videos to 1.06 billion tweets discussing them over six months.The tweeted-video dataset is listed as a contribution and made available by the authors.Source: §1 · pp. 1–2
• HIP extends the Hawkes point process by taking expectations over stochastic event histories to model expected event volumes under continuous external influence.The resulting popularity series closely matches observed view counts, including for videos with complex lifecycles.Source: §1 · pp. 1–2
• The endo-exo map combines endogenous response and exogenous sensitivity to identify videos likely to go viral if promoted and videos unlikely to benefit from promotion.High values on both dimensions indicate viral potential; very low values on either dimension indicate promotion is unlikely to have an effect.Source: §1 · p. 2
• 28.6% lower average percentile error was achieved by HIP than by state-of-the-art methods based on popularity history.The evaluation used 13K+ actively discussed YouTube videos, estimating parameters from each video’s first 90 days and forecasting the next 30 days.Source: §1 · p. 2

2.1 Problem setting: views under promotion

• HIP uses daily views and observed shares or tweets as popularity and promotion measures, respectively, while recognizing that neither promotion source fully captures external discussion.Shares span multiple social platforms but are incomplete within each source; the Twitter feed is nearly complete for promotions on Twitter, and evaluations report the sources separately.Source: §2.1 · p. 2

2.2 Hawkes process for social events

• HIP models viewing events as a Hawkes process in which external promotion and prior views jointly drive the arrival rate, with past events contributing through a decaying power-law kernel.The kernel includes video quality, event magnitude, and a user-influence exponent; its memory-decay parameter controls how quickly reactions to events stop.Source: §2.2 · pp. 2–3

2.4 Estimating HIP from data

• Example fits show HIP tracking complex popularity dynamics: fast memory decay follows stimuli closely, while slow decay accumulates promotion into a rising trend.The music-video example has θ = 5.37 and the news-video example θ = 0.41; the authors state that capturing joint effects enables both short-term dynamics and long-term trends.Source: Figure 2 · p. 3, §2.4 · p. 4

2.3 From Hawkes to HIP

• HIP takes the expected Hawkes event rate over random event histories, yielding an intensity driven by external stimuli and its own history convolved with a power-law memory kernel.This formulation addresses the fact that daily attention volumes are observed instead of individual viewing events, whose full Hawkes estimation is quadratic in the number of events.Source: §2.3 · pp. 3–4

2.4 Estimating HIP from data

• The HIP fitting procedure models unobserved external influence as an initial shock and constant background excitation, then minimizes squared error between observed and modeled daily views.The initial impulse and constant component are used because they require few assumptions about how unobserved influence evolves; adding them improved fitting for many videos.Source: §2.4 · p. 4

2.5 Properties of the HIP

• HIP's linear time-invariant property supports two viral-potential measures: exogenous sensitivity µ and endogenous response Aξ, the total response to one unit of external excitation.Aξ is finite for sub-critical HIPs; the authors find it explains popularity better than other HIP-derived quantities such as C or θ.Source: §2.5 · pp. 4–5

3.1 Dataset construction

• The Twitter API collection covers 2014-05-29 to 2014-12-26 and includes over 81.9 million YouTube videos mentioned in 1.06 billion tweets.The Active dataset retains 13,738 videos across 14 categories with at least 120 days of history and at least 100 tweets and 100 shares by day 120; the first 90 days are used to estimate HIP parameters.Source: §3.1 · p. 5

3.2 The popularity scale

• The popularity scale maps each video’s views or shares to a percentile, reflecting their long-tailed distributions; the tweet scale is similar to the share scale.The mapping functions St(x) and Pt(x) return the percentile for a video's share or view count, respectively, on the scale constructed at time t.Source: §3.2 · pp. 5–6
• Videos from every 30-day popularity bin could jump into the top bins by day 60, although most retained a similar rank or declined slightly.The figure compares views-popularity percentiles at 30 and 60 days after upload.Source: §3.2 · p. 6, Figure 3 · p. 6

The endo-exo map

• The map places endogenous response Aˆξ on the x-axis and exogenous sensitivity µ on the y-axis to explain popularity and identify videos that are not promotable.These are the two quantities defined in Section 2.5.Source: §4 · p. 6

Explaining popularity

• Each unit of exogenous excitation generates µAˆξ events, so nearby videos on the map have similar potential to become popular.Popularity differences between nearby videos are attributed to differences in the exogenous attention they receive.Source: §4 · p. 6
• Video v1 has 4.61x more views than v2 despite similar map positions, explained by receiving 3.22x more exogenous promotions.The map also attributes lower popularity for v3 to lower exogenous sensitivity and for v4 to a smaller endogenous response than v1.Source: §4 · p. 6, Figure 4 · p. 7

What describes the most popular videos?

• Popular Film & Animation videos have higher exogenous sensitivity, while popular Gaming videos have higher endogenous response; other categories can show dense regions high on either factor.The comparison concerns the top 5% of videos in each category; Comedy and News & Politics also show two dense regions.Source: §4 · p. 6, Figure 4 · p. 7

Identifying unpromotable videos

• 549 (∼3.9%) of videos in the Active set are deemed unpromotable, with 63 (∼3.2%) in People & Blogs.A video is considered unpromotable when µAˆξ < 1e-3; one example has µ = 2.88 × 10^-15 and Aˆξ = 1, so each online promotion is expected to generate 0 views.Source: §4 · pp. 6–7, Figure 4 · p. 7

5.1 A video that will go viral

• Video 1PuvXpv0yDM rose from the 5.85th to the 94.9th popularity percentile after receiving 229 shares and gaining 2.42 million views between days 91 and 120.HIP had identified high endogenous response (Aˆξ = 6.94×1072) and exogenous sensitivity (µ = 119.02) after the video received 15,687 views in its first 90 days.Source: §5.1 · pp. 7–8, Figure 5 · p. 8

5.2 Evaluation of forecast

• HIP forecasting assumes future external promotion s[t] is known or estimable, and evaluation uses temporal holdout data to forecast popularity over days 91–120.Performance is measured by absolute error in predicted popularity percentile, which emphasizes ranking videos correctly rather than viewcount error.Source: §5.2 · p. 8

5.3 Forecasting results

• HIP (#shares) achieved a 4.96% average percentile error (median 3%), versus 6.94% (median 3.75%) for MLR (#shares), a 28.6% relative error reduction.The difference was statistically significant (paired t-test p < 0.001) with a medium effect size; HIP’s slight advantage for shares over tweets was not statistically significant at p = 0.001.Source: §5.3 · p. 8, Figure 5 · p. 8
• Among 4,006 videos with a large exogenous shock, HIP (#shares) achieved a 5.11% mean percentile error (median 3.25%), versus 9.24% (median 6.5%) for MLR (#shares).A large shock is defined as the mean plus 100 times the standard deviation of shares during the observed period; HIP can miss when popularity is driven by unrecorded external influence.Source: §5.3 · pp. 8–9
• HIP is causal in the linear-system sense, but does not establish causal effects in the inference sense; Granger tests found no consistent direction between shares and views.Future tweets cannot change past views, but confounding factors remain relevant to causal inference.Source: §5.3 · p. 9

Popularity modeling and prediction

• Unlike single-shock models [23] [28], HIP captures continuous endogenous-exogenous popularity evolution and models attention volumes directly rather than requiring individual event observations.This lets HIP fit complex popularity histories rather than reactions to isolated shocks.Source: §6 · p. 9

Modeling volumes of popularity

• Prior volume models use polynomial-rise/exponential-decay curves, periodic memory, predefined-shape convolutions, or staged progression; HIP forecasts total popularity, recovers parameters, and explains non-stationary variation from external-activity data.The cited volume approaches describe activity shapes in different ways, while HIP addresses their shortcomings through its self-excited Hawkes-process formulation.Source: §6 · p. 9

Influence estimation and maximization

• Unlike influence estimation and maximization [32], HIP measures promotion volume to forecast popularity rather than taking a graph-centric view of user interactions.Influence estimation learns pairwise user-influence probabilities; influence maximization selects promoters to maximize diffusion.Source: §6 · p. 9

• HIP models continued interactions between endogenous virality and exogenous promotion to explain complex, multi-phased popularity dynamics.Source: §7 · p. 9
• Validated on YouTube videos, HIP quantifies each video's virality and promotion sensitivity to identify videos that respond well or poorly to promotion, without platform-dependent assumptions.The model can operate on popularity and promotion series generated by aggregated human behavior, including for content such as webpages, podcasts, or blogs.Source: §7 · p. 9
• HIP models expected influence across users rather than individual influence, omits seasonality, and is limited by diverse online or offline promotion sources that are unobserved or difficult to obtain.The authors identify seasonality components, individual influences, and tracking diverse or unknown sources of exogenous influence as directions for further investigation.Source: §7 · p. 9

1 Details of HIP

• HIP treats events as exogenous arrivals or endogenous responses to earlier events, such as views following a public speech or word-of-mouth diffusion.The model aims to derive the expected event rate as the average response rate across a large network.Source: §1 · p. 11

1.1 Event rate in marked Hawkes processes

• The marked Hawkes process models intensity using external stimulus and past events, whose marks scale influence while a decaying kernel captures temporal memory.The kernel separates into mark influence b(m) = κm^β and temporal decay φ(τ) = (τ + c)^−(1+θ); follower counts approximate social influence, and the estimated user-distribution exponent is α = 2.016.Source: §1.1 · pp. 11–12

1.2 Proof of Theorem 2.1: ξ(t) for unobserved point processes

• The proof derives expected event rate ξ(t) in three stages: counting-process preliminaries, the unmarked case, and then the marked case.It establishes the main-text Theorem 2.1 result by averaging conditional event rates over event histories.Source: §1.2 · p. 12

1.2.1 Preliminaries: event rate, counting process

• For an infinitesimal interval, conditional intensity determines the probability of one counting-process increment, while increments greater than one are very unlikely.The counting-process formulation applies to non-homogeneous Poisson processes, including marked and unmarked Hawkes processes.Source: §1.2.1 · p. 12

1.2.2 Expected event rate for unmarked Hawkes

• For unmarked Hawkes processes, the expected rate averages conditional intensity over event histories and is derived through counting-process increments.The authors describe this expected-intensity integral equation and its analytical derivation as new; Hawkes gave a similar equation for covariance density, not event intensity.Source: §1.2.2 · pp. 12–14

1.2.3 Expected event rate for marked Hawkes

• For marked processes, expected intensity averages over event times and magnitudes, with each event mark independently drawn from the same power-law distribution.Averaging mark influence b(m) yields the modeling constant C, which scales the expected-rate contribution of past events.Source: §1.2.3 · pp. 14–15

1.3 Branching factor and endogenous response

• The branching factor n is the mean number of direct descendants per event; total endogenous response Âξ sums descendants across generations.For n < 1, Âξ = 1/(1−n); for n > 1, it is infinite. The section notes that finite attention precludes indefinite supercritical popularity, while changing external stimuli and endogenous reactions can produce repeated rises and falls.Source: §1.3 · p. 15

1.4 HIP as an LTI system

• HIP is linear time-invariant from external stimulus to expected event rate, enabling additive attribution, proportional scaling, and time-shifted scheduling of promotion responses.Independent excitation sources contribute additively; scaling or shifting stimulus correspondingly scales or shifts the expected event rate.Source: §1.4 · pp. 15–16
• ξ[t] is a superposition of shifted impulse response functions ˆξ[t], scaled by s[τ].Each response is shifted by the corresponding amount.Source: §1.4 · p. 16
• Eq. (33) is Eq. 8 in the main text’s Lemma 2.3.Source: §1.4 · p. 16

2.1 The loss function

• HIP fitting minimizes squared error between observed and modeled popularity to estimate each video’s parameters and unobserved external influence.The fit is recursive: model estimates, rather than observed prior values, feed the modeled series so it reproduces the full observed history.Source: §2.1 · p. 16

2.2 Computing gradients

• Because the objective is non-convex, the authors use L-BFGS with supplied gradients, fit videos in parallel from multiple random starts, and retain the lowest-loss solution.Gradient calculations are iterative, using previous values in each parameter’s derivative series.Source: §2.2 · p. 17

2.3 Adding an L2 regularizer

• L2 regularization is applied to γ, η, µ, and C, using unregularized-fit reference values to normalize their weights; θ and c are left unregularized.The regularization strength is selected by log-scale line search as a percentage of the unregularized loss.Source: §2.3 · p. 17

2.4 Properties of the model estimates

• Given θ and c, the loss is convex in the remaining linear parameters, but HIP fitting estimates both linear and nonlinear parameters for interpretability and performance reasons.The section contrasts this choice with fixing nonlinear parameters, as in Zhao et al., and cites a performance advantage for estimating both parameter types in prior work.Source: §2.4 · p. 18
• The optimizer is guaranteed only to reach a local minimum; eight random starts improve solution quality, while convergence rates are unknown and the model assumes stationarity over time and network regions.The final result is the run with the lowest loss.Source: §2.4 · p. 18

3 Data

• Between 2014-05-29 and 2014-12-26, the Twitter crawl yielded 1,061,661,379 tweets referencing 81,915,174 distinct YouTube videos.The crawl used YouTube-related Twitter keywords; YouTube metadata and daily views, shares, and tweets were collected for each video.Source: §3 · p. 18

3.1 The 5Mo and Active datasets

• The 5Mo forecasting subset contains 16,417,622 videos with at least 60 days of popularity history; the Active dataset contains 13,738 videos meeting activity and observation-window thresholds.More than half of videos lacked publicly available popularity histories. Active excludes six categories with under 1% representation; its largest four categories account for over 70% of videos, with Music exceeding 25%.Source: §3.1 · pp. 18–19, Table 1 · p. 19

3.2 The popularity scale over time

• At 30 and 60 days after upload, the popularity scale uses 40 bins in 2.5-percentile increments, with each bin containing about 41,000 or more 5Mo videos.Bins are defined by view counts at each video age and marked by their maximum popularity percentile.Source: §3.2 · p. 19
• The popularity distribution is long-tailed: the least-popular 15% of videos each have fewer than 10 views, while the top 5% span almost two orders of magnitude.Middle bins from the 17.5th through 85th percentiles have view counts within 1.5 times of one another; the overall scale retains a similar shape at 30 and 60 days.Source: §3.2 · p. 19
• Videos from any popularity bin can reach the top bins between 30 and 60 days, while Active videos fall in the top 5% of 5Mo popularity percentiles.Most videos retain a similar rank or experience a slight rank decrease over that period.Source: §3.2 · p. 19

4.1 Behavior across groups of videos: categories and channels

• Game-recording videos are generally more popular than the reporter’s news videos, which the paper attributes to their higher exogenous sensitivity µ.The endo-exo map visualizes these two channels’ videos by popularity percentile.Source: §4.1 · p. 20, Figure 9 · p. 20

The effect of the external influence is not equal

• Nonprofits & Activism videos receive more than the median number of shares despite having fewer-than-average views.The category comparison covers views, shares, and tweets in the Active dataset.Source: The effect of the external influence… · p. 21, Figure 10 · p. 21

Observed versus unobserved external influences

• Gaming, Comedy, and Entertainment videos have higher initial impulse γ and constant exogenous stimuli η, potentially reflecting activity outside Twitter or YouTube sharing.The paper gives gaming-focused subreddits and forums as examples of external activity not captured in observed sharing.Source: Observed versus unobserved external… · p. 21, Figure 10 · p. 21

Exogenous sensitivity and endogenous response

• Comedy, Gaming, and Sport appear particularly sensitive to external influence, while Comedy, Entertainment, and Gaming have higher-than-median endogenous responses.The combination of high sensitivity and endogenous response helps explain Comedy and Gaming’s high views despite relatively low sharing; Nonprofits & Activism has lower-than-median values for both and low popularity despite high sharing.Source: Exogenous sensitivity and endogenous… · p. 21, Figure 10 · p. 21

4.2 Categories of longer versus shorter memory

• Music videos have lower mean θ than all videos (14.95 versus 15.94), while Nonprofits & Activism and News & Politics have higher means (17.56 and 19.45), indicating category differences in memory decay.All categories and the overall population show long-tailed θ distributions with peak density around 3.35; smaller θ corresponds to slower decay.Source: §4.2 · pp. 21–22, Figure 11 · p. 22

4.3 Model parameters and popularity

• Videos with high exogenous sensitivity µ tend to be more popular, while videos with high memory exponent θ and lower endogenous response tend to be less popular.Low-µ videos cluster at low popularity; high-θ videos are forgotten more quickly, whereas lower-θ videos with higher endogenous response tend to be more popular.Source: §4.3 · p. 22
• On the endo-exo map, the top 5% most popular videos skew toward high µ and high endogenous response, while category-specific patterns include two distinct routes for Comedy.Comedy’s popular-video subgroup has one heat center associated with higher exogenous sensitivity and another with higher endogenous propagation.Source: §4.3 · pp. 22–23, Figure 12 · p. 23

4.4 Potential causal connection between the views, tweets and shares series

• Granger causality was detected for no more than 6% of Active-set videos, and the analysis found no clear causal relation among views, shares, and tweets.The test used a p-value threshold below 10^-3; views Granger-caused shares for 833 videos.Source: §4.4 · pp. 23–24, Table 2 · p. 24

5 Popularity forecasting and comparison to baseline

• The model is fit on each video’s first 90 days, then forecasts views for days 91–120 against MLR baselines with and without exogenous inputs.The first 75 days fit model parameters and the last 15 tune the regularizer; 3.5% of videos meeting the stated burst-outlier criterion are removed from MLR training.Source: §5 · p. 24

5.1 Additional results

• HIP forecasts 87% of videos within 10% error, versus 78% for MLR; HIP (#shares) has a 3% median error, versus 3.75% for MLR (#shares).The 87% and 78% figures refer to the same 10% error threshold; MLR (#shares) adds only marginal improvement over MLR.Source: §5.1 · p. 24, Figure 13 · p. 25

5.2 Comparing performance

• HIP and MLR differ significantly for both stimuli sources, with p-values of ∼10^-151 for #shares and ∼10^-95 for #tweets and small, non-negligible effect sizes.The analysis uses paired t-tests to compare algorithms on the same individuals and measures effect size with Cohen’s d, alongside p-values for the 13,738-video Active dataset.Source: §5.2 · pp. 25–26, Table 3 · p. 26
• Comparisons show no significant performance difference between #shares and #tweets for either algorithm; their time series are highly correlated for most videos.The first test’s p-value is ∼10^-7, which the authors consider likely an artifact of the large sample size; both source comparisons have negligible Cohen’s d.Source: §5.2 · pp. 25–26, Table 3 · p. 26

5.3 Forecasting performance on difficult videos

• For 4,006 videos with high exogenous shocks, HIP (#shares) has a 3.25% median error, versus 6.5% for MLR (#shares).High shock means at least one test-period stimulus value exceeds the training mean plus 10 times the training variance; HIP performs similarly to its performance on the full Active dataset.Source: §5.3 · p. 26

• For the exogenous set, median absolute forecasting error was 3.25% for Hawkes (#shares) versus 6.5% for MLR (#shares).Source: Figure 15 · p. 27
• The illustrated Brazilian-politics video received considerable attention in October/November 2015, more than three months after upload, then slid back into obscurity after December 2015.Source: Figure 15 · p. 27

Autos & Vehicles: all

• Autos & Vehicles, People & Blogs, and Pets & Animals are each labeled “top 5%.”Source: Autos & Vehicles: all · p. 28
• Figure 16 compares all-video and most-popular-5% density distributions across Active dataset categories, with remaining categories continued in Figure 17.Gaming and Film & Animation were already shown in Main Text Figure 4.Source: Figure 16 · p. 28, Figure 17 · p. 29