Balancing Trial and Reorder: A Hybrid Sequential Transformer-GBDT Ranker for On-Demand Delivery

1 Introduction

• Wolt operates across 30+ countries and more than 1,000 cities, serving restaurant, grocery, and retail delivery.Users encounter store selections through multiple surfaces, including multi-list and single ranked-list formats.Source: §1 · p. 1
• Delivery ranking must balance trial discovery and reorder relevance while selecting from local, dynamic candidate sets constrained by availability and operations.The paper frames this as a constrained multi-objective problem implemented with empirically tuned per-session weighting.Source: §1 · p. 1
• Purchase histories encode predictive signals for future decisions because user behavior is sequential.Source: §1 · p. 1
• UVR pairs a bidirectional transformer for sequential user modeling with a GBDT ranker integrating contextual and content features.The system is trained per country across stores and domains, replacing the previous production ranking stack with one unified model.Source: §1 · p. 1

2 Related Work

• Transformer recommenders use causal or bidirectional attention, with later work advancing their objectives and sampling strategies.The cited literature also applies transformers with Semantic IDs to retrieval and search.Source: §2 · p. 1
• Delivery recommendation must address local candidate sets, real-time availability, and multi-stakeholder objectives, which remain underexplored relative to general e-commerce.MTGR exemplifies target-aware generative multi-candidate scoring; UVR’s classifier scores the candidate slate in one pass with a non-target-aware bidirectional encoder.Source: §2 · p. 2
• Trial–reorder balancing has been studied mainly in music and video streaming, although delivery users also alternate between trusted-store reorders and new-store trials.Source: §2 · p. 2
• Cross-domain recommenders transfer knowledge across domains to mitigate cold-start and improve recommendation quality.UVR’s unified model is intended to let behavior from one vertical inform another and to handle user cold-start across domains.Source: §2 · p. 2
• UVR integrates the four research threads into a single production system.Source: §2 · p. 2

3 Problem Formulation

• The framework treats each user’s time-ordered lifetime purchases as history and uses the subsequence before the reference time.The formulation frames personalized store ranking as sequential recommendation with explicit time conditioning.Source: §3 · p. 2
• At request time, UVR ranks the candidate stores eligible for a user’s location and time so the purchased store receives a high rank.The candidate set is determined by vicinity and operational constraints; retrieval of that set is out of scope.Source: §3 · p. 2
• The formulation assumes single-purchase sessions; sessions with multiple purchases use the purchased store with the best production rank, while sessions without purchases are discarded.Source: §3 · p. 2
• Purchases are partitioned into cold-start, trial, and reorder types: cold-start users have no prior purchase history, while trial and reorder distinguish first from repeat purchases at a store.Source: §3 · p. 2
• Purchase is the primary target, clicks enter as an auxiliary indicator, and the framework can extend to other interaction types.Candidate retrieval is outside the scope of this work and depends on delivery-area size, store availability, and courier availability.Source: §3 · p. 2

4.1 Classifier: User Sequence Transformer for Next-Purchase Prediction

• The encoder-only classifier predicts the next store from purchase sequences using bidirectional attention, with request time, location, and domain visible while the target store and product line are masked.The model adopts BERT4Rec’s bidirectional-attention backbone but masks only the most recent purchase.Source: §4.1 · p. 3

4.1.1 Embedding Layer.

• Each sequence step combines six categorical fields and a continuous time gap, discretizing location to H3 cells and adding a dedicated domain embedding at the target step.The additive post-projection domain signal sharpened conditioning; FiLM-style featurewise modulation showed no benefit over the simpler additive scheme.Source: §4.1.1 · pp. 3–4

4.1.2 Encoder Layer. 5

• The encoder uses four pre-LayerNorm transformer layers, and the target step attends to all historical positions without a causal mask.Sequences shorter than the context length are left-padded, and padding positions are excluded from attention.Source: §4.1.2 · p. 4

4.1.3 Prediction Layer.

• The prediction head trains per-store logits with weighted, label-smoothed softmax cross-entropy, using weights ordered new > cold-start > reorder.For cold-start users, the single masked target step yields a domain-, location-, and time-conditioned prior over stores.Source: §4.1.3 · p. 4

4.2 Ranker: GBDT for Listwise Store Ranking

• The CatBoost GBDT ranker scores each candidate store using transformer logits, store-content, contextual, and user–store interaction features; structurally different restaurant and retail features motivate separate domain rankers.Source: §4.2 · p. 4

4.3 Coupling and Cross-Domain Transfer

• Cross-domain transfer combines chronologically interleaved histories, per-position product-line embeddings, and target-step domain conditioning, so restaurant history informs retail predictions and vice versa.Source: §4.3 · p. 4

4.4 Model Training and Validation

• UVR is trained as a country-specific model and retrained daily; classifier training precedes ranker training, with disjoint data slices to prevent validation and test sessions from leaking into classifier training.The daily cadence addresses accumulating unseen stores because store, H3-cell, and product-line embeddings cover values observed during training.Source: §4.4 · p. 5

4.4.1 Classifier.

• The transformer is trained for 15 epochs with AdamW, label smoothing ε = 0.2, and sample weights ordered w_new > w_cs > w_rec.Source: §4.4.1 · p. 5
• Classifier checkpoints maximize trial MRR on new-store validation targets, with reorder and cold-start MRR as guardrails; validation is decomposed to monitor these target types separately.Sequences referencing unseen store, H3-cell, or product-line IDs are moved into training, and histories are capped at the most recent M purchases.Source: §4.4.1 · p. 5

4.4.2 Ranker.

• Ranker training uses purchase sessions, normalizing multi-store purchases to the purchased store with the smallest production rank; evaluation uses the full backend slate, including unseen stores.Training negatives come from impressed-but-not-purchased stores, which are competitive alternatives already selected for display.Source: §4.4.2 · pp. 5–6
• For each session, one classifier forward pass supplies per-candidate logits as ranker features, with the target step carrying request context and a domain-specific product-line mask.Source: §4.4.2 · p. 6
• The ranker uses a click-aware YetiRankPairwise NDCG objective and trial-weighted sessions, with w_new > w_rec; its weight ratio is empirically tuned to balance trial quality against bounded reorder regression.Purchases outrank clicks, and clicks outrank no interaction in the row-label scheme.Source: §4.4.2 · p. 6

• Evaluation measures MRR separately for cold-start, new, and reorder targets across benchmark, production, classifier, and ranker scores on the held-out final day.The benchmark combines a binary reorder indicator with store-relative popularity and otherwise uses no further personalized signal.Source: §4.5 · p. 6
• At trial-weighted settings, the ranker can fall below both the classifier and production ranking on reorder MRR.Source: §4.5 · p. 6
• A ranker is blocked from rollout unless new-segment MRR exceeds benchmark MRR at 95% confidence; reorder MRR raises a warning but does not block deployment.Source: §4.5 · p. 7

• Daily training jobs run for each country, and inference sequences are written to the online feature store to synchronize training and serving features.In this setting, daily updates were sufficient because purchases are relatively sparse signals.Source: §5 · p. 7
• Content, contextual, and user–store interaction features are computed in separate batch pipelines and published every few hours to daily, depending on upstream availability.Source: §5 · p. 7
• Serving retrieves features, computes transformer classifier logits on CPUs, and applies the GBDT ranker, with approximately 60 ms end-to-end p99 latency.Source: §5 · p. 7

6.1 Baselines

• UVR was compared with four legacy rankers: three restaurant models and one retail GBDT; offline tests focused on restaurants, while V1 and V2 were restaurant-only.The reported offline country subset represented more than 50% of Wolt’s worldwide order volume.Source: §6.1 · p. 7, §6 · p. 7

6.2 Offline Testing

• The GBDT ranker improved over the classifier in all 12 country–purchase-type cells, with new-purchase gains of +7.5% to +19.8% in every country.The authors attribute the ranker’s gains to complementary content and contextual features beyond the sequence signal.Source: §6.2 · p. 7, Table 4 · p. 7
• V2 amplified V1’s new-purchase MRR gains in all six countries, while its classifier outperformed its ranker on recurring purchases in five of six.Examples of new-purchase gains were ISR +19.7% → +30.3% and DEU +19.8% → +29.0%; individual table cells are directional given run-to-run variability.Source: §6.2 · p. 7, Table 4 · p. 7
• The chosen setting maximized new-purchase MRR uplift subject to nonnegative new uplift and a recurring-purchase regression no worse than −15%; DEU had no qualifying setting.The selection operationalizes the stated trade-off constraint.Source: §6.2 · p. 7

6.3 Online A/B Testing

• +5.5% Merchant Trial Rate and +0.16% Global CVR for V1; V2 added +0.45% Merchant Trial Rate, and V3 added +1.31% Retail Merchant Trial Rate.V3 showed no significant change in restaurant-domain trial rate or blended Global CVR; the offline reorder regressions also left Global CVR statistically unchanged.Source: Table 5 · p. 8, §6.3 · p. 8, §6.4 · p. 8
• The A/B tests used 50:50 user-level random splits; V1 and V2 ran for two weeks in ten large markets, while V3 ran for three weeks across all operating countries.The longer V3 test reflected the larger sample needed for retail-domain significance.Source: §6.3 · p. 8

6.4 Discussion and Limitations

• The V3 result does not isolate whether retail benefits from cross-domain transfer or from adopting a transformer–GBDT stack; the relevant ablations were not controlled.The available comparisons differ in features, losses, and training data.Source: §6.4 · p. 8
• UVR uses purchase histories delayed by up to one day, limits the classifier to stores seen in training, and excludes interaction signals such as clicks and browsing.For new stores, ranking therefore relies on the GBDT’s content features alone.Source: §6.4 · p. 8

7 Conclusion

• Three A/B tests reported +5.5% Merchant Trial Rate and +0.16% Global CVR for V1, +0.45% additional Merchant Trial Rate for V2, and +1.31% Retail Merchant Trial Rate for V3.V3 retired the last domain-specific legacy ranker.Source: §7 · p. 8
• At the deployed operating point, UVR gained +12% to +30% offline trial MRR over production while losing reorder MRR in five of six countries, without that trade-off surfacing in Global CVR.The per-session weight ratio and sampled-negative count scalarize the constrained multi-objective problem.Source: §7 · p. 8
• Future work targets candidate-store conditioning, inductive content representations for cold-start mitigation, and real-time interaction signals for short-term intent.Source: §7 · p. 8

• The ablation held the classifier fixed and tested three wnew/wrec settings (1.5, 2.0, 3.0) across 50 max_pairs settings per regime, sharing country, day, and random seed.Source: A Trial vs. Reorder Trade-off · p. 10, Figure 3 · p. 10
• Weight ratios trace a Pareto-style frontier: 1.5 favors reorder with slight trial regression, 2.0 modestly lifts both, and 3.0 favors trial at reorder’s cost.Source: A Trial vs. Reorder Trade-off · p. 10, Figure 3 · p. 10
• Within each weighting cluster, increasing max_pairs consistently shifts results toward stronger reorder and weaker trial.Source: A Trial vs. Reorder Trade-off · p. 10

• Reorder MRR rises sharply up to about 30 purchases, then plateaus as maximum input sequence length increases.Source: B MRR vs. Maximum Sequence Length · p. 10, Figure 4 · p. 10
• Trial and cold-start MRR are largely insensitive to sequence length, motivating a moderate training sequence cap.Source: B MRR vs. Maximum Sequence Length · p. 10
• Figure 4 compares classifier MRR across the rec, cs, and new validation slices against maximum input sequence length.Source: Figure 4 · p. 10