PaperlayerView exact revision on arXiv

Source-linked AI summary

Hierarchical Exponential-Gaussian Mixtures for Watch-Time Distribution Prediction

Sofia Gulevskaia, Mikhail Trapeznikov, Aleksandr Poslavsky, Alexander D'yakonov

arXiv:2608.23356v1cs.IRcs.LGstat.ML

TL;DR

Short-video WT prediction requires modeling engagement distributions that are heterogeneous and difficult for point objectives, while deployment demands stable, interpretable ranking heads. The paper diagnoses EGMN’s failure modes and proposes HEGM with hierarchical decomposition and stabilization changes. HEGM improves ranking and threshold-event prediction across datasets and achieves statistically significant production engagement lifts.

  • Problem

    WT distributions are near-zero-inflated, long-tailed, and multimodal, while industrial systems require reliable distributional predictions for engagement modeling.

  • Method

    HEGM reformulates EGMN using hierarchical skip-watch decomposition, structured initialization, KL-based variance regularization, and removal of the forced Gaussian shift and entropy regularizer.

  • Results

    HEGM achieves superior XAUC across all evaluated datasets, improves threshold-event prediction, and produces statistically significant engagement lifts in production.

  • Takeaways & Limitations

    Distributional WT modeling can be accurate, stable, interpretable, and deployable when its mixture structure is aligned with user behavior.

  • Takeaways & Limitations

    NLL is not used for model selection or as a primary metric because variance collapse can make its behavior unsuitable for that role.

Abstract

from arXiv · show

Accurate watch-time (WT) prediction is an important requirement for short-video recommendations. Yet WT distributions are near-zero-inflated, long-tailed and multimodal. The recent Exponential-Gaussian Mixture Network (EGMN) models the full conditional WT distribution rather than a single point estimate and achieves state-of-the-art performance. Our large-scale reproduction study reveals that EGMN is vulnerable to variance collapse, component redundancy, and inactive components. We propose a Hierarchical Exponential-Gaussian Mixture (HEGM) model that addresses these failure modes through a hierarchical skip-watch decomposition, KL-based variance regularization, structured initialization, removing the forced Gaussian shift and the entropy regularizer. Across public and large-scale industrial datasets, HEGM improves ranking accuracy and threshold-event prediction, while maintaining competitive point-estimation accuracy and substantially improving mixture stability and interpretability. A 1.5-month production A/B test confirms statistically significant engagement lifts. Our code and models are publicly released at https://github.com/rw404/HEGM.

1 Introduction

Short-video watch-time is a dense engagement signal, but its near-zero inflation, skew, long tails, multimodality, and biases make scalar prediction inadequate. HEGM addresses EGMN’s industrial-scale fragility while preserving lightweight distributional modeling for production recommendation.

  • Watch-time is a dense continuous engagement signal in autoplay feeds, where click-through rate is less informative than viewing duration.
  • WT distributions combine near-zero inflation, right skewness, long tails, multimodality, duration bias, and ranking-oriented evaluation requirements.
  • Distributional prediction estimates the full conditional density, supporting expected WT, threshold probabilities, completion probabilities, and uncertainty-aware decisions.
  • EGMN models WT with one exponential and several Gaussian components, offering an analytically clear and easily integrated distributional head.
  • HEGM preserves EGMN’s advantages while improving reliability and interpretability through hierarchical decomposition, regularization, initialization, and parameterization changes.
  • Across public and industrial benchmarks, HEGM outperforms EGMN and other baselines in ranking quality and threshold-event prediction, with production validation and released resources.

2 Problem Formulation and Evaluation Metrics

The paper formulates watch-time prediction from impression features and evaluates both point and conditional-distribution predictions using metrics aligned with ranking, accuracy, density quality, and threshold events. NLL is treated cautiously because variance collapse can make it unreliable for model selection.

  • 2.1 Short-Video Watch-Time Prediction: Each impression contains a user, video, context, duration, and observed WT, while features are constructed from these variables for engagement prediction.
  • 2.1 Short-Video Watch-Time Prediction: Global target scaling preserves absolute ranking information, unlike per-user, per-item, or per-duration normalization.
  • 2.2 Point and Distributional Prediction: Point regression maps each instance to a scalar and uses efficient losses, but discards skewness, multimodality, and uncertainty.
  • 2.3 Evaluation Metrics: Evaluation covers practical dimensions including ranking consistency, original-scale point accuracy, distributional quality, and threshold-event prediction.
  • 2.3.1 Ranking Quality: XAUC estimates whether predicted scores correctly order comparable interaction pairs with different ground-truth WTs.
  • 2.3.2 Point accuracy: MAE and MSE are computed on the original WT scale using the predicted expectation for distributional models.
  • 2.3.3 Distributional Quality: NLL is the canonical continuous-density score, but the paper avoids using it for model selection or as a primary metric because of variance collapse.
  • 2.3.4 Threshold-event prediction: ROC AUC evaluates binary predictions for WT thresholds such as deep watch or percentage completion.

3 Related Work

Prior work progresses from scalar regression and discretization toward conditional distribution estimation, but each family has limitations in fidelity, flexibility, complexity, or deployment suitability. EGMN is the closest precursor, and HEGM reformulates it for more robust production use.

  • Scalar regression is efficient but compresses heterogeneous, skewed, and multimodal WT distributions into one statistic under assumptions violated by real data.
  • Discretization methods improve ranking-oriented prediction but depend on bucket or tree design and lose fine-grained continuous WT structure.
  • Quantile, adversarial, behavior-driven, and prototype-transport methods provide alternatives for uncertainty, continuation modeling, or calibration without uniformly supplying a continuous density.
  • Generative approaches are expressive but add architectural complexity, create train-inference mismatch, and are less suitable as lightweight drop-in ranking heads.
  • EGMN models WT as one exponential component plus several Gaussian components, matching quick skips and multiple engaged-watch regimes.
  • HEGM builds on EGMN by changing its parameterization and optimization while retaining low integration cost and analytic clarity for production backbones.

4 Preliminaries: EGMN

EGMN models watch time as a conditional mixture distribution, using an exponential component for quick skips and Gaussian components for engaged-watch patterns. Its architecture predicts mixture parameters from a backbone representation and trains with likelihood, regression, and entropy-based objectives.

  • EGMN treats watch time as a conditional random variable to represent quick skips, partial views, completions, and replays.
  • The conditional density combines one exponential component with K Gaussian components whose weights are nonnegative and sum to one.
  • The exponential models coarse-grained skewness and near-zero skips, while Gaussians capture fine-grained engaged-watch patterns.
  • A backbone maps features x to latent representation h, and separate output heads parameterize mixture components including Gaussian standard deviations.
  • EGMN combines NLL, MAE regression, and entropy regularization, with entropy encouraging diversity among mixture weights.

5 Limitations and Design Rationale

The reproduction study identifies instability and underutilization in EGMN mixtures, including variance collapse, redundant Gaussian components, inactive components, and occasional loss of multimodality. These pathologies can reduce interpretability and production ranking performance.

  • Grouped average densities are used because individual predicted densities are noisy, making cohort- and category-level visualizations smoother and more interpretable.
  • The reproduction and industrial evaluation expose several failure modes inherent to EGMN.
  • 5.1 Gaussian Variance Collapse: Variance collapse drives Gaussian standard deviations toward zero, producing needle-like likelihood spikes, memorization, unstable uncertainty estimates, and poor interpretability.
  • 5.2 Component Redundancy & Inactivity: Gaussian components may merge, inactive weights may approach zero, or the mixture may become nearly pure exponential, limiting effective capacity and multimodality.
  • Basic EGMN underperforms scalar MSE regression on production XAUC, reaching 0.6585 versus 0.7070, with results sensitive to initialization.

6 Proposed Method: HEGM

HEGM reorganizes watch-time prediction into hierarchical skip and engaged-watch stages, then stabilizes Gaussian components through structured initialization and a duration-conditioned variance prior. It also removes EGMN’s forced Gaussian shift and entropy regularization to improve interpretability and avoid empirically unhelpful constraints.

  • 6.1 Hierarchical Skip–Watch Decomposition: HEGM uses a sigmoid gate to estimate skip probability before modeling engaged-watch behavior with a Gaussian mixture.
  • 6.1 Hierarchical Skip–Watch Decomposition: The hierarchical decomposition separates quick skips from active engagement, providing an interpretable distinction useful for downstream business analytics.
  • 6.2 Global Normalization and Structured Initialization: Global normalization and uniformly spaced Gaussian-mean initialization reduce component merging and stabilize early training.
  • 6.2 Global Normalization and Structured Initialization: HEGM initializes Gaussian standard deviations at 1.5/K and the exponential rate from a short-watch prior of 0.05, approximately 9 seconds industrially.
  • Unlike EGMN, HEGM directly parameterizes Gaussian means and removes entropy regularization because the forced shift and entropy term did not improve performance empirically.
  • A KL variance prior anchors component variances to duration-conditioned empirical scales while discouraging spikes and overly diffuse components without constraining means.

7 Experiments

The experiments compare HEGM with regression and distributional baselines across public and industrial datasets under a shared evaluation protocol. Models are selected by validation XAUC using chronological splits and common backbones.

  • Experiments use KuaiRec, VK-LSVD, and a proprietary Industrial dataset from a production short-video platform.
  • All datasets are split chronologically to avoid look-ahead bias and reflect production scenarios.
  • Evaluation protocol: All heads share the same underlying backbones to avoid implementation and tuning differences disadvantaging the comparison.
  • Baselines: HEGM is evaluated against MSE-VR, MAE-VR, CREAD, and the original EGMN.
  • Training: Mixture models tune K over {3, 6, 9, 12}, while KL and regression weights are selected from predefined grids.
  • Evaluation protocol: Validation XAUC is the primary selection metric because the production ranker is optimized for ordering quality.

7.2 Main Results and Convergence Behavior

HEGM delivers the strongest ranking performance across datasets while remaining competitive on point-estimation metrics and supporting distribution-based threshold prediction. Its convergence can require longer training, and KL regularization is especially useful for tail-sensitive threshold decisions.

  • Main results: HEGM achieves superior XAUC across all datasets compared with all baselines.
  • Main results: 12.97 vs 24.14: HEGM achieves the best Industrial MAE with a substantial margin over CREAD.
  • Main results: HEGM performs best on public-dataset MSE but underperforms MSE-VR on Industrial, while MSE regression is inferior in XAUC (0.7070 →0.7188).
  • Main results: HEGM often outperforms EGMN in NLL, but NLL is excluded from the main table because variance collapse makes it unstable to interpret.
  • Convergence behavior: HEGM overtakes CREAD only after approximately 20 epochs, indicating that short early-stopping schedules may underestimate mixture-density heads.
  • Threshold prediction: At the 60 seconds threshold, HEGM reaches 0.8903 ROC AUC versus 0.8356 for EGMN.
  • Threshold prediction: KL regularization more consistently improves threshold evaluation by stabilizing component variances and producing more reliable tail densities.

7.4 Ablation Studies

Ablations show that structured initialization is central to HEGM’s ranking performance, while KL regularization addresses variance collapse and instability as mixtures become larger.

  • Ablation results: Removing structured initialization sharply reduces XAUC from 0.7188 to 0.6841.
  • Ablation results: Flat softmax, the forced Gaussian shift, and entropy regularization each degrade the reported metrics when reintroduced.
  • Variance collapse: For K = 3 and K = 6, HEGM eliminates variance collapse at 0%, whereas EGMN collapses in 5%–13% of components.
  • Variance collapse: At K = 12, unregularized HEGM reaches a 47.88% collapse rate under ϵ = 10^-3.
  • Variance collapse: Adding KL variance regularization eliminates collapse in the over-parameterized K = 12 setting.
  • Training stability: With K = 8, KL regularization reduces XAUC fluctuations, whereas both variants behave similarly at K = 3.

8 Online Deployment and A/B Testing

HEGM was deployed as a drop-in replacement for the production watch-time head and evaluated through a 1.5-month randomized A/B test. It improved engagement metrics while adding limited serving latency.

  • Deployment: HEGM replaced the proprietary multi-task watch-time head without changing retrieval or feature generation.
  • A/B testing: The user-level randomized A/B test allocated 5% of live traffic each to HEGM and the production baseline for 1.5 months.
  • A/B testing: Statistical significance was assessed with CUPED-adjusted user-level t-tests, with all reported metrics satisfying p < 0.05.
  • Online results: +9.26%: session depth increased from 28.52 to 31.16 videos per session.
  • Online results: All primary engagement metrics improved, while total view time remained largely unchanged and likes did not show a comparable lift.
  • Resource impact: Median latency increased by 1.22 ms, while p99 rose from 7.9 ms to 8.2 ms, below the 30 ms serving constraint.

9 Conclusion

The paper positions HEGM as a reliable and interpretable approach to distributional WT modeling, developed through diagnosis, redesign, ablation, validation, and production testing. It also identifies causal debiasing, component-family choice, and probabilistic evaluation as remaining limitations.

  • The work frames distributional WT modeling as a broader shift beyond scalar prediction and fixed discretization.
  • The research cycle combines empirical diagnosis, hypothesis-driven redesign, ablation analysis, offline validation, and live production testing.
  • The authors identify three limitations: HEGM is not an explicit causal debiasing method, alternative component families may better match WT support, and probabilistic quality needs broader evaluation.
  • HEGM is presented as accurate, stable, interpretable, and deployable when its mixture structure aligns with user behavior.
Loading 2608.23356v1…