Source-linked AI summary
Decomposing Staleness in Recommender Systems: A Dual-Filter Framework for Supersession and Decay
Di Bai, Feng Han, Zhenwei Tang, Jintao Liu, Luoshu Wang, Jialu Liu
TL;DR
Stale recommendations arise when newer content supersedes prior coverage or when an item's relevance decays over time, while existing heuristics detect these mechanisms poorly. SDF uses complementary learned filters for supersession and decay before ranking, reducing user-filed staleness reports by 54.9% over two years in Google Discover.
Problem
Existing age-based and engagement-based heuristics poorly detect supersession and relevance decay, leaving stale recommendations in large-scale feeds.
Method
SDF combines a relational pairwise model for supersession with a predicted traffic ratio model for intrinsic decay, filtering candidates upstream of ranking.
Results
54.9%: user-filed staleness reports fell over a two-year Google Discover deployment, including 64.0% for supersession and 34.4% for decay.
Takeaways & Limitations
SDF supports dual-mechanism staleness filtering at industrial scale while improving user experience and serving efficiency.
Takeaways & Limitations
SDF misses some gradually decaying items and does not address staleness from content users have already seen elsewhere.
Abstract
from arXiv · showhide
Stale recommendations are a pervasive challenge and a leading source of user complaints on large-scale content platforms. Items lose relevance through two primary mechanisms: supersession, where emerging updates render prior coverage stale, and relevance decay, where an item's informational value naturally diminishes over its lifecycle. Traditional countermeasures serve as crude proxies: age cutoffs poorly reflect actual relevance loss, while engagement heuristics rely on lagging signals, broadly exposing users to stale content before the system adapts. We present SDF (Supersession-Decay Filtering), a staleness filtering system fully deployed in Google Discover, a personalized recommendation feed with hundreds of millions of daily and billions of monthly active users. SDF targets both mechanisms with complementary filters, each powered by a learned model: a relational staleness model that detects supersession between item pairs, and a predicted traffic ratio (PTR) model that forecasts relevance decay from the item's content, trained on lifetime visit traffic. Applied via disjunction upstream of the ranking stage, SDF prunes stale candidates, measurably reducing downstream serving costs. Online experiments demonstrate that these filters significantly reduce the prevalence of stale content while improving user engagement. Over a two-year production deployment, user-filed staleness reports (in-product user feedback) declined by 54.9% relative to the pre-deployment baseline, establishing SDF as a robust and scalable paradigm for resolving content staleness at industrial scale.
1 Introduction
SDF treats recommendation staleness as two distinct mechanisms—supersession and relevance decay—and proactively filters both before computation-heavy ranking. Deployed in Google Discover, it reduced user-filed staleness reports by 54.9% over two years.
- Staleness mechanisms: Supersession makes items stale when newly arriving updates advance the underlying topic, creating a relational rather than absolute staleness signal.Examples include an announcement of Queen Elizabeth II’s passing superseding an earlier bulletin and Ohtani’s signing superseding speculation about offers.
- Staleness mechanisms: Relevance decay reduces an item’s informational value over its lifecycle based on its own content and intended lifespan.Time-sensitive material, such as a guide to tonight’s meteor shower, can lose usefulness sharply without any superseding item.
- Motivation: Traditional defenses are inadequate because ranking demotion and age cutoffs either continue serving stale items or remove useful evergreen content.Engagement heuristics also introduce structural lag by learning staleness only after repeatedly exposing users to degraded experiences.
- SDF framework: SDF combines a relational supersession filter with an intrinsic PTR decay filter and applies their disjunction upstream of ranking.The relational model uses pairwise classification with rationale supervision, while PTR uses multimodal content and lifetime visit traffic to forecast future relevance decay.
- Results: 54.9% reduction in user-filed staleness reports was achieved across a two-year Google Discover production deployment.The reduction was 64.0% for supersession and 34.4% for decay.
2 Related Work
Prior recommender-systems research has mainly addressed staleness indirectly through popularity decay, using time-based modeling and empirical content-lifecycle analysis. Industrial systems commonly manage freshness with static age cutoffs and engagement-based candidate-filtering heuristics.
- Staleness modeling in recommender systems: Staleness research has primarily modeled popularity decay indirectly rather than detecting staleness directly.The passage frames indirect popularity-decay modeling as the dominant research paradigm.
- Staleness modeling in recommender systems: Time-based approaches add explicit temporal features, personalized click-through-rate decay, or recency–relevance tuning.Examples include TimeSVD++, personalized time-decay for click-through rate, and tuning recency against relevance.
- Staleness modeling in recommender systems: Another research direction empirically characterizes content lifecycles using news shelf-life curves and popularity-trajectory prediction.The passage identifies social-media activity as a basis for news shelf-life curves.
- Industrial practices on staleness and corpus filtering: Industrial recommender systems typically combine static age cutoffs with engagement-based heuristics to manage candidate freshness.The passage presents this combination as recurring across diverse platforms.
3 Methodology
SDF decomposes staleness into supersession and intrinsic relevance decay, using complementary learned filters composed by disjunction to remove stale candidates before ranking. The relational filter detects when newer topic-related items supersede existing items, while PTR removes items whose predicted lifetime traffic has largely accumulated.
- Dual-filter composition: SDF maintains the candidate set A by removing items when either the relational supersession filter or intrinsic decay filter fires.The two filters are composed by disjunction, so an item remains only when neither mechanism flags it.
- Supersession detection: The relational filter predicts whether a newly arriving item d′ ∈ R(d) renders an existing item d stale, then removes d from A when any pairwise prediction fires.This task targets supersession between items sharing a major topic, such as an event or entity.
- Intrinsic decay detection: The PTR filter estimates the fraction of an item’s lifetime traffic accumulated by age τ and removes it when g(d_t,τ) ≥ θ.The threshold indicates that most lifetime traffic has already accumulated and further serving is expected to yield diminishing returns.
- Relational model training: The pairwise supersession model is trained by distilling LLM-teacher labels into a compact student model using synthesized, balanced item-pair data.The pipeline selects item pairs and obtains class-and-rationale annotations from multiple independent LLM samples aggregated by majority vote.
- PTR model training: PTR is trained on anonymized search-click logs, using active user selections as a proxy for relevance rather than publisher-biased raw page views.In production, lifetime traffic is operationalized with a fixed 30-day reference window and discrete horizons including 12h, 1d, 3d, 5d, 7d, 10d, and 14d.
4 Experiments
SDF is evaluated through offline filter tests, engagement-neutral online A/B experiments, and a two-year deployment analysis. Results cover staleness classification, user-impact metrics, and operational outcomes, while also identifying limitations of each filter.
- Evaluation design: SDF evaluation spans per-filter offline tests, online A/B testing with prevalence delta, and two-year deployment outcomes for user reports and serving costs.These three axes correspond to Sections 4.1–4.3.
- Offline evaluation: The relational staleness model is evaluated with Accuracy, Precision, Recall, and F1, using F1 as the primary metric.Table 1 ablates discriminative versus generative supervision and auxiliary NLI training.
- Offline evaluation: PTR is evaluated as a binary stale-item classifier at threshold θ using held-out items labeled independently by five human raters.Pairwise evaluation is inappropriate because decay is expressed through lifetime traffic curves rather than fixed pairwise labels.
- Online evaluation: Online experiments use differential weekly sampling across control and experiment arms, with five-rater ordinal judgments binarized when at least three raters mark an item stale.The composite metric combines supersession and decay staleness for launch decisions.
- Online evaluation: Engagement-neutral A/B tests compare SDF and its component filters against age-cutoff and engagement-cliff heuristics, excluding aggressively shrinking candidate sets or degrading engagement.Negative Δstale indicates lower staleness in the promoted set than the demoted set.
- Long-term impact: −0.16 ± 0.07% user dismissal rate, +0.13 ± 0.04% feed diversity, and +0.26 ± 0.18% feed positive engagement were observed versus a two-month joint holdback.The reported ± ranges are 95% confidence intervals.
5 Conclusion
SDF reframes recommendation staleness as a joint property of an item and its changing environment. It addresses supersession and lifecycle relevance decay with complementary learned models.
- 5 Conclusion: SDF models staleness through two complementary dimensions: relational supersession by newer items and intrinsic relevance decay over an item’s lifecycle.The framework treats continuously shifting relevance as central to recommender-system staleness.
- 5 Conclusion: An LLM-distilled pairwise model addresses the relational dimension by capturing supersession between items.
- 5 Conclusion: A multimodal predicted traffic ratio model addresses the intrinsic dimension by capturing relevance decay across an item’s own lifecycle.
GenAI Usage Disclosure
The authors used Google Gemini to refine the manuscript, manually reviewed all AI-assisted revisions, and retained full responsibility for the final work’s accuracy and integrity.
- GenAI Usage Disclosure: Google Gemini assisted manuscript refinement, while the authors manually reviewed every AI-assisted revision and assumed full responsibility for the final work.The authors specifically retained responsibility for the accuracy and integrity of the final manuscript.