EGA-V1: Unifying Online Advertising with End-to-End Learning

1 INTRODUCTION

• MCA stages have divergent objectives and capacities, while independent-CTR ranking misses how ad permutations and contextual interactions shape user preferences.The resulting prediction discrepancies can propagate through the pipeline and degrade displayed ads.Source: §1 · p. 1
• EGA-V1 unifies online advertising ranking in one model for LBS, narrowing the candidate set to ads within the same city, about 10^5 candidates.The paper says this enables fine-grained contextual modeling across the candidate space and eliminates conflicts between pipeline-stage goals.Source: §1 · p. 2
• The proposed system addresses feature-processing costs, quadratic attention bottlenecks, and advertising-specific objectives such as auction dynamics, bid optimization, and advertiser goals.These are identified as challenges for generative advertising systems.Source: §1 · p. 2
• EGA-V1 combines a decoupled Hybrid Feature Service, cluster-attention RecFormer, non-autoregressive AucFormer, and multi-stage training for platform objectives and auction constraints.The stated goals include lower feature-processing overhead and faster inference without compromising platform objectives.Source: §1 · p. 2
• CTR improved 5.2%, RPM 13.6%, and advertiser ROI 3.1% over state-of-the-art MCAs in offline experiments and large-scale A/B tests.The reported improvements were statistically significant.Source: §1 · p. 2

2 RELATED WORKS

• A typical MCA uses recall, pre-ranking, ranking, and auction stages, with models and latency constraints differing across stages.Recall retrieves candidates efficiently, pre-ranking approximates ranking under tighter latency, and ranking estimates CTR with richer features.Source: §2 · p. 2
• Capacity disparities can create prediction discrepancies between pre-ranking and ranking, while separable-CTR ranking misses mutual influences among candidate ads.The passage names DSSM and COLD for pre-ranking and DIN and SIM for ranking.Source: §2 · p. 2
• Because early filtering shrinks the candidate space before auction, cascaded systems restrict auction mechanisms’ ability to model complete ad externalities.The auction stage allocates ads according to platform revenue objectives.Source: §2 · p. 3

3.1 Preliminaries

• The task is to select a revenue-maximizing winning ad sequence from candidates while satisfying incentive-compatibility and individual-rationality constraints.A request has N competing advertisers and K ad slots, with K < N.Source: §3.1 · p. 3

3.2 Hybrid Feature Service

• HFS stores infrequently updated candidate-ad features locally, retrieves frequently updated user and context features once per request, and broadcasts them across candidates.It retains core categorical ad features; omitted numerical and cross-features are intended to be recovered by later interaction modules.Source: §3.2 · p. 4

3.3 RecFormer

• GCF uses clustered keys and values so original candidate queries attend to N_c aggregated representations rather than all N candidates.The learnable cluster matrix groups keys and values; this reduces the stated complexity from O(N^2d) to O(NN_cd), with N_c ≪ N.Source: §3.3 · pp. 4–5
• MIF performs cross-interactions at intermediate transformer blocks and adds context attention, which sends attention signals from user behaviors to the candidate-ad set.For the stated assumptions m = 10, m_c = 2, N = 10^5, L = 10^3, and N_c = 100, MIF requires 1/10^4 the FLOPs of early fusion and half the FLOPs of late fusion.Source: §3.3 · p. 5

3.4 AucFormer

• AucFormer pairs a non-autoregressive generator that predicts allocation probabilities across slots with a permutation-aware evaluator that estimates engagement metrics such as pCTR and pCVR.Its payment network computes payment rates with a sigmoid and outputs payments by multiplying those rates by bids.Source: §3.4 · pp. 5–6

4 TRAINING AND OPTIMIZATION

• Pre-training aligns EGA-V1 with engagement signals such as clicks and purchases, while post-training optimizes responses to auction feedback and payment under economic constraints.Requests combine sampled unexposed ads with exposed ads; labels use current-request clicks for exposed ads and platform-wide click behavior for unexposed ads, with purchases and reviews also usable.Source: §4 · p. 6, §4.1 · p. 6

4.1 Pre-training

• The pre-training dataset samples unexposed ads by popularity while retaining long-tail coverage, alongside exposed ads from actual impressions.The objective uses current-request clicks for exposed ads and users’ broader platform click behavior for unexposed ads.Source: §4.1 · p. 6

4.2 Post-training

• Post-training trains a permutation-aware reward model on exposed ads, then uses auction-feedback reinforcement learning to optimize the generated sequence for marginal platform revenue.During this phase, RecFormer is frozen and optimization focuses on AucFormer; the payment network is optimized with a Lagrangian dual formulation balancing revenue and IC constraints.Source: §4.2 · pp. 6–7

5.1.1 Dataset.

• The industrial dataset contains 200 million requests from more than 2 million users across nearly 10 million ads, with the last 14 days held out for testing.Logs span April–October 2024; testing uses all candidate ads per sample, with N=100,000 and K=5.Source: §5.1.1 · p. 7

5.1.2 Evaluation Metrics.

• Offline evaluation uses Recall@50 and AUC for pre-ranking and ranking, plus replay-estimated expected CTR and revenue; online metrics are RPM, CTR, and ROI.The IC metric Ψ is empirically tested through counterfactual bid perturbations.Source: §5.1.2 · p. 7

5.1.3 Baselines.

• The baselines are MCA, implemented with SASREC, DSSM, DIN, and GSP, and FS-LTR, which trains with relabeled data across stages.The FS-LTR setup uses the MCA architecture with its sample relabeling technique.Source: §5.1.3 · p. 7

5.1.4 Hyperparameters.

• EGA-V1 hyperparameters were selected by grid search, while MCA and FS-LTR settings follow the cited prior setup.The reported configuration uses adaptive clustering, learning rate 10^-3, batch size 128, and 128 adaptive clusters.Source: §5.1.4 · p. 7

5.2 Offline Experiments

• Recall@50 reaches 0.513, a 20.4% lift over FS-LTR; AUC is 0.754 (+1.48%), eCTR improves +8.3%, eRPM is 217.1 (11.4% above FS-LTR), and Ψ is 2.3% versus 9.1% for FS-LTR and 9.4% for MCA.Table 1 reports the offline comparison; the text says EGA-V1 improves over the baselines in AUC, Recall@50, eCTR, eRPM, and Ψ.Source: §5.2 · pp. 7–8, Table 1 · p. 8

5.3 Ablation Study

• Removing GCF, MIF, or AucFormer worsens online results; the reported findings associate these components with externality modeling, deep user-behavior modeling, and auction-preference alignment, respectively.The AucFormer/RLAF variant comparison shows an obvious performance gap, and the authors report improved ad-sequence generation and overall performance with those components.Source: §5.3 · p. 8, Table 2 · p. 8

5.4 Cross Features Analysis

• With 18 of 94 cross-features removed, replacing MIF with late fusion degrades AUC by 1.0% and increases deviation by 2.6%.Removing context-attention and target-attention individually reduces AUC by 0.9% and 1.1%, respectively; the authors report their combined contribution as synergistic.Source: §5.4 · p. 8, Table 3 · p. 8

5.5 Scaling Law

• Increasing RecFormer and AucFormer block counts improved performance progressively, with diminishing returns beyond a threshold.The authors attribute the plateau likely to fixed data scales and sequence lengths; exploring larger datasets and longer sequences was beyond scope due to resource constraints.Source: §5.5 · pp. 8–9
• Higher FLOPs correlated with better recommendation accuracy in the tested range, although marginal gains diminished at higher computational costs.Source: §5.5 · p. 9, Figure 5 · p. 9

5.6 Online Results

• 5.2% CTR, 13.6% RPM, and 3.1% ROI improvements over the deployed MCA were observed in the online A/B test.The test ran November 18–24, 2024, on a low-traffic Meituan ad slot.Source: §5.6 · p. 9, Table 4 · p. 9
• Hundreds of times more candidate ads were processed with only a 5 ms average response-time increase (2.2% relatively) per request.Source: §5.6 · p. 9

• EGA-V1 unifies the entire ad ranking and allocation pipeline as one model.Source: §6 · p. 9
• Its innovations include decoupled user and ad feature processing, cluster attention for interests and externalities, and bi-stage training aligned with user preference and platform objectives.Source: §6 · p. 9
• Experiments on public and industrial datasets demonstrate EGA-V1’s superiority over state-of-the-art approaches.Source: §6 · p. 9

• For a standard Transformer block, the analysis gives total FLOPs as 4L^2d + 24Ld^2.Source: §A.1 · p. 10
• The MCA estimate includes dual-tower pre-ranking over candidates and a Transformer-based ranking stage using early fusion.Source: §A.1 · p. 10
• The comparison calculates EGA-V1 complexity across Global-Cluster Former, Mid-Interest Interest-Former, and AucFormer modules.Source: §A.1 · p. 10
• Both MCA and EGA-V1 computational complexities are proportional to the number of candidates N.Source: §A.1 · p. 10
• With α = 0.033, the reported ratio of proportions is 0.97, and the authors say the slight difference yields a significant FLOPs gap as candidate counts increase.Source: §A.1 · p. 10

• Ex-post regret quantifies the potential gain an advertiser could obtain by untruthfully reporting its bid, enabling differentiable incentive-compatibility constraints.Source: §A.2 · p. 10
• The auction design minimizes expected negative revenue subject to empirical ex-post regret remaining zero for each ad.The empirical estimate uses sampled valuations.Source: §A.2 · p. 10
• With the payment-network loss, EGA-V1 ensures incentive compatibility approximately.Source: §A.2 · p. 10