Source-linked AI summary
Exploratory creative pre-ranking: Hierarchical empirical-Bayes Thompson sampling for real-time creative optimization
Jiale Han, Ajith Moparthi, Dan Xie, Xiaowu Dai
TL;DR
Creative pre-ranking must learn from sparse, cold-start feedback while scoring every recalled creative within request latency. The method combines a data-calibrated hierarchy, segment-level estimates from marginal counters, and Thompson sampling; on replayed Roblox traffic, it lifts predicted PTR by 8.14% versus 2.80% for an equally sized random shortlist.
Problem
Creative pools are large and feedback per creative is sparse, so pre-ranking must learn under cold start while scoring every recalled creative cheaply within the request.
Source: §1 · pp. 2–3Method
A segment-level Thompson-sampling pre-ranker combines marginal-counter estimates with a hierarchical posterior whose parent borrowing is calibrated from historical logs.
Source: §1 · p. 3, §3.1 · p. 7Results
8.14% predicted play-through rate versus 2.80% for an equally sized random shortlist; the method forwards over 40% fewer creatives than the full pool and adds 0.2% to serving time.
Source: §1 · p. 3Takeaways & Limitations
After an A/B test on live traffic, the method now serves in production at Roblox.
Source: §1 · p. 3The segment-rate reconstruction assumes attribute lifts combine linearly, with one coefficient per attribute.
Source: §3.2 · p. 9
Abstract
from arXiv · showhide
Dynamic creative optimization decides which creative of an ad to render for each request. Recent work places that choice before ad ranking, but relies on deterministic point predictors with no calibrated uncertainty. AI-generated content has made creatives nearly free, while feedback per creative remains sparse and selection faces a tight latency budget. We introduce a creative pre-ranker that scores every recalled creative and shortlists each campaign's pool before the per-user ranker runs. Creatives sit in a hierarchy of audience segment, advertised item, campaign and creative, so a cold creative falls back on what its parent levels have earned, and how much it borrows from its parents is set in closed form from historical logs, with one parameter per level and nothing tuned. To stay within the latency budget, the pre-ranker predicts for an audience segment rather than an individual user, from per-attribute counters. It selects by a single Thompson draw per creative, balancing exploration and exploitation so that a new creative still has a chance to be learned. Replaying Roblox Sponsored-ads traffic over campaigns with competing creatives, the shortlist lifts predicted play-through rate by 8.14% against 2.80% for an equally sized random shortlist, forwards more than 40% fewer creatives than the full pool, and adds only 0.2% to serving time. After an A/B test on live traffic, the method now serves in production at Roblox.
1 Introduction
Creative pre-ranking must learn from sparse, cold-start feedback while scoring every recalled creative cheaply enough for the request. This section introduces a hierarchical, audience-level Thompson-sampling pre-ranker and reports replay results on Roblox traffic.
1 Introduction
- Creative pools have grown as AI-generated content makes creatives inexpensive, while each creative-by-segment cell receives only a handful of impressions.The pre-ranker must therefore learn under sparse feedback while remaining cheap enough to score the larger-than-ad-pool creative set within a request.Source: §1 · pp. 2–3
- The proposed pre-ranker predicts for an audience segment, borrows strength from hierarchy parents, and uses calibrated uncertainty to explore cold creatives.Its hierarchy’s borrowing strength is set in closed form from historical logs, with one parameter per level and nothing tuned.Source: §1 · p. 3
- Marginal counters provide a cheap audience-level estimate, with the combining rule fitted offline and exact when attributes are independent or redundant.This avoids maintaining counters for every combination of categorical attributes.Source: §1 · p. 3
- 8.14% predicted play-through rate versus 2.80% for an equally sized random shortlist; the method forwards over 40% fewer creatives and adds 0.2% to serving time.After an A/B test on live traffic, it serves in production at Roblox.Source: §1 · p. 3
1.1 Related Work
- Earlier pre-ranking work uses deterministic point predictors without calibrated uncertainty, while this paper contributes an exploratory pre-ranker based on a hierarchical posterior over marginal counters.The paper frames its contribution as a production-ready instantiation of empirical Bayes shrinkage and Thompson sampling, not a new shrinkage method or regret analysis.Source: §1.1 · p. 4
2 Problem Setting
The system must reduce recalled creative pools before ad ranking without discarding creative-level engagement signal. The paper defines a segment-level PTR pre-ranker and models rates through a hierarchy whose parent estimates can support data-sparse leaves.
2.1 Creative Selection Before Ad Ranking
- Each advertised universe may contain multiple campaigns, each with many creative renderings, and the system first recalls eligible ads for the request’s categorical-attribute segment.PTR is the probability that a user enters the advertised game.Source: §2.1 · pp. 4–5
- The pre-ranker shortlists each recalled campaign’s creatives before the ranker, predicting segment-level PTR so user-level resolution remains downstream.The ranker’s cost grows with the candidate set it receives, while deterministic selection can prevent unchosen creatives from accumulating evidence.Source: §2.1 · p. 5
2.2 A Hierarchical Random-Effects Model of PTR
- PTR is decomposed into mean-zero shifts along the hierarchy from global root to segment, universe, campaign, and creative, allowing sparse nodes to draw on parent information.Conditioned on a true rate, a node’s plays are Binomial.Source: §2.2 · pp. 5–6
3 Main Results
Section 3 builds a real-time creative pre-ranker from hierarchical empirical-Bayes estimates, reconstructed from marginal counters and sampled with Thompson sampling. Its design calibrates shrinkage from historical data, estimates audience-level rates cheaply, and uses posterior draws to form campaign shortlists.
3.1 Empirical-Bayes Shrinkage with a Data-Calibrated Concentration
- The MSE-optimal shrinkage weight is calibrated from historical logs, with one prior strength per hierarchy level and no tuned parameter.The Beta working prior preserves the optimal shrinkage and gives a conjugate posterior; its concentration is the node’s impression count at the equal-weight crossover.Source: §3.1 · pp. 6–8
3.2 A Real-Time Posterior from Marginal Counters
- Single-attribute counters replace infeasible joint node-by-segment counters, and offline-fitted pooling weights combine their marginal lifts into audience-specific rates.The weights account for correlated targeting attributes; the estimate is reconstructed at request time from counters that are cheap to maintain and query.Source: §3.2 · pp. 8–10
- Weights fit on universe cells cut held-out creative-cell reconstruction mean absolute error by about a third relative to (15).The weights are shared across levels because targeting-attribute overlap is a property of the request stream, while dense cells support fitting.Source: §3.2 · p. 10
3.3 Exploratory Creative Pre-Ranking
- Thompson sampling scores each creative with a fresh posterior draw, giving uncertain creatives a chance to be tried while segment-specific posteriors guide exploration.Campaign quotas scale with pool size, and round-robin allocation under a global cap prevents larger creative pools from gaining shortlist share merely by uploading more.Source: §3.3 · pp. 11–12
4 Experiments
The experiments replay production Roblox Sponsored-ads traffic using the counters available at request time, comparing shortlist rules on predicted play-through rate and serving cost. The hierarchical Thompson-sampling shortlist substantially outperforms equal-slot random selection, with small added serving cost; results also examine quota and latency sensitivity.
4.1 Data
- The replay samples a few hundred production requests and caps each recalled set at 500 campaigns, retaining each campaign’s full creative pool.A sampled request contains more than 1,000 creatives on average, about 2.6 per campaign; 16.9% of candidates have neff = 0.Source: §4.1 · p. 13, §4 · p. 13
4.2 Setup and Compared Methods
- The five compared rules vary whether they select randomly or by Thompson sampling and whether they use one slot or a pool-size-based quota.Uniform and TS-One share an allocation, as do RR-Random and TS-RR; slot allocations depend on pool sizes, ρ, and B, not scores.Source: §4.2 · pp. 13–14
4.3 Evaluation Metrics
- Shortlist reward is the best per-user predicted play-through rate among its creatives, averaged over campaigns with at least two creatives.Predictions come from the deployed ranking model; single-creative campaigns are excluded because every rule returns the same creative there.Source: §4.3 · p. 14
4.5 Results
- +8.14% predicted play-through rate for TS-RR versus +2.80% for RR-Random, with TS-RR recovering 44.3% of the gap to forwarding everything.The comparison uses the same slot allocation; TS-RR closes 44.3% of the distance to forwarding the full pool.Source: §4.5 · p. 15, Table 1 · p. 15
- The full pre-ranker takes 2.2 ms, while forwarding 100 additional creatives adds 6.4 ms of ranking cost.TS-RR and RR-Random take 41.6 ms and 41.5 ms, respectively, a 0.2% difference.Source: §4.5 · p. 15, Table 2 · p. 16
- Under a hard latency cap, TS-RR returns +3.34% while RR-Random returns essentially nothing; at ρ = 1.0, both return +10.31%.The sensitivity sweep uses another deployed ranking model.Source: §4.5 · p. 16, Table 3 · p. 16
5 Conclusion
The conclusion presents a lightweight, hierarchical creative pre-ranker that estimates parent borrowing from logs and selects using Thompson sampling. On replayed Roblox traffic, it recovers nearly three times the gain of an equally sized random shortlist at 0.2% additional serving cost; after a live A/B test, it serves in production.
5 Conclusion
- The method estimates parent borrowing from logs, reconstructs per-segment posteriors from per-attribute counters, and selects with a single Thompson draw.The conclusion notes that joint counters over attribute combinations grow exponentially and are hard to maintain.Source: §5 · p. 17
- On replayed Roblox traffic, the method recovers close to three times the improvement of an equally sized random shortlist at 0.2% additional serving cost.After an A/B test on live traffic, it serves in production at Roblox.Source: §5 · p. 17
- The construction is presented as applicable where a ranker sees a shortlist drawn from sparse, hierarchically organized items, including cold-start items in web search and recommendation.Learning the residual from linear log-odds reconstruction and allocating slots by expected information gain are identified as next steps.Source: §5 · p. 17
A.1 Proof of Theorem 1
The appendix proves the shrinkage estimator by decomposing its squared error and then derives an expected-scatter identity for children sharing a parent. The scatter result supports the variance estimation used in the pooling procedure.
A.1 Proof of Theorem 1
- Conditioning on the latent rate makes the empirical rate’s error mean zero, eliminating the cross term in the shrinkage estimator’s squared-error expansion.The proof then minimizes a convex quadratic in the weight and substitutes the resulting weight into the estimator.Source: §A.1 · p. 22
A.2 The Scatter Decomposition
- The expected impression-weighted scatter separates into a term proportional to between-child variance and a binomial-noise term.The derivation uses the weighted identity for Q and independence of children to evaluate the variance of their weighted mean.Source: §A.2 · p. 23
A.3 Proof of Corollary 1
The appendix derives a parent-level variance estimator from child scatter, then analyzes how per-attribute log-odds reconstruction behaves under independent or duplicated attributes. It also notes an iterative fitting choice for pooling weights.
A.3 Proof of Corollary 1
- The known-mean scatter estimator is unbiased; replacing unknown parent means with estimates and clipping at zero adds bias that vanishes at the impression volumes used to fit the prior.The implemented estimator is consistent for the shared variance.Source: §A.3 · pp. 23–24
A.4 Independence and Duplication
- With independent attributes, each per-attribute lift isolates its centered effect; duplicated attributes instead produce double counting unless their group is weighted by 1/J.The stated derivation assumes additive joint log-odds and buckets measured without error by exposure-weighted log-odds averages.Source: §A.4 · pp. 24–25
A.5 Fitting the Pooling Weights
- The pooling-weight fit iterates rather than stopping at the one-step least-squares version because saturated cells can push weakly identified attributes outside [0,1].The passage describes this issue for cells with extreme targets and ample impression weight.Source: §A.5 · p. 25
Appendix B. Experimental Details
Serving combines a periodically published hierarchical table with each creative’s rapidly updated counters. The replay reconstructs those inputs as they were available at request time and summarizes its sample using rates rather than totals.
B.1 Serve-Time State
- Table 4 summarizes the replay sample as shares and per-request or per-campaign rates; a pool is a request-campaign pair, contested when it has at least two recalled creatives.Production scoring and replay both use Algorithm 1’s online steps, reading the published hierarchy and creative counters.Source: §B.1 · p. 25, Table 4 · p. 26
- The hierarchical table stores node-level rates, shrunk marginal lifts, concentrations, and pooling weights fitted on several days of impressions.It is republished regularly; the creative’s short-window counters update with every impression and are the only fast-moving posterior input.Source: §B.1 · p. 26
- The replay uses each candidate’s counters and published hierarchical row as available at that request, not a later state.Logged ranker predictions provide the rewards.Source: §B.1 · p. 26
B.2 Operating Point and Implementation Details
The reported operating point uses a shortlist cap chosen to bind while preserving at least one slot per campaign. Evaluation averages repeated Thompson draws and request-clustered bootstrap samples, with timing measured on a reference implementation and hardware setup.
B.2 Operating Point and Implementation Details
- At ρ = 0.8, B = 600 lies between the campaign-count and uncapped-quota bounds, so reduction binds while every campaign retains at least one slot.The round robin therefore redistributes only second and later slots; deployment sets B from the ranker’s latency budget.Source: §B.2 · p. 26
- Quality averages 200 Thompson-draw replications and uses 5,000 request-clustered bootstrap resamples; timings average 25 repeats per request.The NumPy reference implementation bypasses the per-campaign selection loop for score-independent rules.Source: §B.2 · p. 27
- Timing uses single-process CPU measurements with ranking models limited to 8 threads, while production serves them on GPU.The comparison relies on how model cost scales with forwarded creatives; ranker timing covers forward passes on preassembled tensors.Source: §B.2 · p. 27
B.3 Production Ranking Models and Robustness
The shortlist’s quality ordering persists under a second deployed ranking model with a substantially different input structure. The main evaluation uses L2, which processes many per-candidate feature paths and deduplicates outputs by advertised universe.
B.3 Production Ranking Models and Robustness
- The main evaluation uses L2, a deployed deep click-through-rate model whose many per-candidate feature paths make cost grow with shortlist size.L2 deduplicates to at most one creative per advertised universe, so reward (23) is a maximum over shortlist members.Source: §B.3 · p. 27
- Under L1, every method keeps its Table 1 ordering, and Algorithm 1 closes essentially the same gap: 44.0% against 44.3%.Table 5 reports robustness under L1 with the setup matching Table 1.Source: §B.3 · p. 27
- L1 is used as a second quality check rather than a substitute for the exploratory pre-ranker; its sole per-candidate input is an ad-identifier hash.Its other inputs are request- and user-level personalization history and context.Source: §B.3 · p. 27
B.4 The Price of Exploring
Ranking by a posterior draw gives up some replay lift relative to ranking by posterior means, while the authors argue that the frozen replay understates the value of exploration. Table 5 separately reports robustness under L1 using the Table 1 setup.
B.4 The Price of Exploring
- The posterior-draw rule lifts predicted play-through rate by 8.14%, versus +9.51% for Mean-RR; exploration costs 1.37 points of lift.Both rules beat the score-free round robin threefold, and the authors attribute the gain to the posterior rather than the draw.Source: §B.4 · p. 27
- Table 5 reports robustness under L1 using the same setup as Table 1.Source: Table 5 · p. 28
- Because the replay freezes each creative’s posterior regardless of whether it was shown, the authors call +8.14% a lower bound on the sampled rule’s online value.In online serving, impressions would update the evidence available to the posterior.Source: §B.4 · p. 28