Source-linked AI summary
Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising
Dongbo Xi, Zhen Chen, Peng Yan, Yinger Zhang, Yongchun Zhu, Fuzhen Zhuang, Yu Chen
TL;DR
Long, imbalanced audience conversion paths make end-to-end conversion prediction difficult, and existing multi-task methods do not explicitly model their sequential dependence. AITM adaptively transfers information between stages and adds a Behavioral Expectation Calibrator; offline and online experiments report significant improvement over state-of-the-art baselines.
Problem
Audience conversion paths can be long, with increasingly sparse positive feedback and delayed activation labels, while explicitly modeling sequential dependence remains a challenge for end-to-end conversion prediction.
Source: §1 · pp. 1–2Method
AITM uses an Adaptive Information Transfer module to pass information between conversion stages and a Behavioral Expectation Calibrator in its loss function.
Source: §1 · p. 2Results
Takeaways & Limitations
The calibrator is intended to make model results better satisfy production constraints and improve end-to-end conversion identification.
Source: §1 · p. 2The framework assumes sequential conversion labels, with each later step possible only after the previous step; other stage combinations are treated as illegal.
Source: §1 · p. 2
Abstract
from arXiv · showhide
In most real-world large-scale online applications (e.g., e-commerce or finance), customer acquisition is usually a multi-step conversion process of audiences. For example, an impression->click->purchase process is usually performed of audiences for e-commerce platforms. However, it is more difficult to acquire customers in financial advertising (e.g., credit card advertising) than in traditional advertising. On the one hand, the audience multi-step conversion path is longer. On the other hand, the positive feedback is sparser (class imbalance) step by step, and it is difficult to obtain the final positive feedback due to the delayed feedback of activation. Multi-task learning is a typical solution in this direction. While considerable multi-task efforts have been made in this direction, a long-standing challenge is how to explicitly model the long-path sequential dependence among audience multi-step conversions for improving the end-to-end conversion. In this paper, we propose an Adaptive Information Transfer Multi-task (AITM) framework, which models the sequential dependence among audience multi-step conversions via the Adaptive Information Transfer (AIT) module. The AIT module can adaptively learn what and how much information to transfer for different conversion stages. Besides, by combining the Behavioral Expectation Calibrator in the loss function, the AITM framework can yield more accurate end-to-end conversion identification. The proposed framework is deployed in Meituan app, which utilizes it to real-timely show a banner to the audience with a high end-to-end conversion rate for Meituan Co-Branded Credit Cards. Offline experimental results on both industrial and public real-world datasets clearly demonstrate that the proposed framework achieves significantly better performance compared with state-of-the-art baselines.
1 INTRODUCTION
Financial advertising involves long, imbalanced conversion paths, with activation feedback delayed and difficult to obtain. The paper proposes AITM to model sequential dependence across these steps and reports improved offline and online performance.
1 INTRODUCTION
- Financial conversion paths can run from impression through click, application, and approval to activation, whose feedback is delayed and difficult to obtain.Activation is typically measured within 14 days, and its label has more severe class imbalance.Source: §1 · p. 2
- The conversion process assumes sequential dependence: a later step may occur only after the preceding step, and other stage combinations are treated as illegal.The paper describes five audience stages and focuses on the last two as desired outcomes.Source: §1 · p. 2, Figure 1 · p. 2
- Existing multi-task approaches either share bottom-layer experts or transfer task probabilities, but do not explicitly model sequential dependence among conversion steps.The probability-transfer approach also omits richer vector representations and can spread errors when a task probability is inaccurate.Source: §1 · p. 2
- AITM’s Adaptive Information Transfer module learns what and how much information to pass between conversion stages, while a Behavioral Expectation Calibrator improves end-to-end identification and aligns results with production constraints.The module uses information from earlier tasks, which have richer positive samples, to help later tasks with class imbalance.Source: §1 · p. 2
- Offline experiments on industrial and public real-life datasets and online experiments show significant improvement over state-of-the-art baselines.The framework is also reported as deployed in the Meituan app.Source: §1 · p. 2
2 RELATED WORK
The related work groups multi-task learning approaches into expert sharing and probability transfer. It identifies a gap in modeling sequential dependence while retaining useful representations near task outputs.
2 RELATED WORK
- Expert-Bottom models share experts across tasks and use separate task towers, but the towers do not exchange their richer, higher-level information.The related-work discussion includes MMoE, PLE, and MoSE among these approaches.Source: §2 · p. 3
- Probability-Transfer models relate task outputs through probabilities, but scalar transfer discards richer vector representations and can propagate errors across tasks.ESMM transfers probabilities by multiplying post-impression CTR and post-click CVR to obtain CTCVR.Source: §2 · p. 3
- Other approaches use tensor factorization, tensor normal priors, or attention, but are not designed to model sequential dependence among audience conversions.The cited approaches include,, and, respectively.Source: §2 · p. 3
3 THE MTL RANKING SYSTEM IN MEITUAN APP
Meituan’s credit-card ranking system models four conversion tasks, excluding the passive impression step, and prioritizes approval and activation. It ranks business offers and selects conversion objectives to accommodate different banks’ needs.
3 THE MTL RANKING SYSTEM IN MEITUAN APP
- The system models click, application, approval, and activation; approval and activation are main tasks, while click and application are auxiliary tasks.The last two end-to-end objectives are impression-to-approval and impression-to-activation; earlier tasks provide more positive samples that can help with class imbalance.Source: §3 · p. 3
- A selector chooses different conversion objectives for banks: start-up banks may prioritize card issuance, while mature banks may prioritize activation.The stated priorities reflect banks’ different stages of business development.Source: §3 · p. 4
- The multi-business ranking system scores business banners using predicted conversion probability and weights representing audience, business, and audience-business value, then shows the highest-scoring banner.Source: §3 · p. 4
4 METHODOLOGY
AITM frames audience conversions as an ordered sequence and predicts end-to-end probabilities at each stage. It transfers information between adjacent stages and calibrates predictions to respect the expected probability ordering.
4.1 Problem Formulation
- The formulation treats completed conversions as an ordered sequence (y_1 ≥ ... ≥ y_T) and predicts the probability of completing all steps through each stage from x.The framework predicts an end-to-end conversion probability for each step.Source: §4.1 · p. 4
4.2 Adaptive Information Transfer Multi-task (AITM) framework
- AITM combines shared feature embeddings and task-specific Towers with an adjacent-stage module that learns transferred content and adaptively weights it against each task’s original representation.Its attention-like weighting uses Value, Query, and Key projections; shared embeddings can exploit richer earlier-task positives and reduce parameters.Source: §4.2 · pp. 4–5
4.3 Behavioral Expectation Calibrator and Joint Opratortimization for MTL
- Training minimizes cross-entropy across tasks plus a Behavioral Expectation Calibrator penalty that discourages later-stage end-to-end probabilities from exceeding earlier ones.The penalty is zero when the ordering holds and is intended to improve alignment with production constraints and end-to-end conversion identification.Source: §4.3 · p. 5
5 EXPERIMENTS
The experiments compare AITM with single-task and multi-task baselines on industrial and public datasets, then examine online performance, information transfer, task count, and hyperparameter sensitivity.
5.1 Datasets
- The industrial dataset contains four conversion tasks; the public Ali-CCP dataset contains click and purchase, and evaluation reports end-to-end AUC on the focused main tasks.Industrial data are split chronologically, with activation negatives downsampled for training while the test set retains the real distribution; offline metrics average five runs.Source: §5.1 · p. 5, §5.2 · p. 5
5.3 Baseline Methods
- Baselines span LightGBM, a single-task MLP, probability-transfer ESMM/ESM2, and Expert-Bottom methods OMoE, MMoE, and PLE.PLE separates task-shared and task-specific experts.Source: §5.3 · p. 6
5.4 Performance Comparison
- AITM significantly improves on state-of-the-art baselines, while PLE is the strongest baseline on both industrial end-to-end tasks.The reported offline results focus on approval and activation AUC; Table 2 presents mean±std AUC for both datasets and marks statistically significant improvements.Source: §5.4.1 · p. 6, Table 2 · p. 6
- On the public dataset, AITM performs best on the purchase task, where positive samples comprise 0.02%; multi-task models show little advantage on click.The authors attribute the limited click-task improvement to there being only two tasks and relatively abundant click positives.Source: §5.4.1 · p. 6, Table 2 · p. 6
- Compared with MLP, AITM raises online impression-to-approval conversion by 25.00% and impression-to-activation conversion by 42.11%.These are two-week A/B comparisons on the same traffic; all feedback takes four weeks to arrive.Source: §5.4.2 · p. 7, Table 3 · p. 7
5.5 Ablation Study
- Under low prediction confidence, transferred-plus-original information improves on original information, and AIT further improves on that combination; more tasks also improve performance.The task-count experiments add stages from activation alone through click-to-activation, with additional positive-sample and transferred information.Source: §5.5 · p. 8, Figure 5 · p. 8
5.6 Case Study
- When a former-stage label is 0, transfer weights are near 1; for former-stage positives, weights rise as the latter task's predictions worsen.For latter-stage positives, little information is transferred, consistent with the latter task relying mainly on itself.Source: §5.6 · p. 8, Figure 6 · p. 8
5.7 Hype-parameter Study
- AITM is not very sensitive to hyperparameters; the authors set embedding dimension d=5, calibrator strength α=0.6, and activation positive-sample proportion λ=1%.The calibrator improves overall performance despite fluctuations across tasks; excessive activation-negative downsampling loses negative information and approval positives.Source: §5.7 · pp. 8–9
6 CONCLUSION
AITM models sequential dependence in audience multi-step conversions through adaptive information transfer. Offline and online experiments show significant improvement over state-of-the-art baselines.
6 CONCLUSION
- AITM models sequential dependence among audience multi-step conversions using an Adaptive Information Transfer module.The AIT module learns what and how much information to transfer across conversion stages.Source: §6 · p. 9
- Combining AIT with a Behavioral Expectation Calibrator in the loss function supports learning information transfer across conversion stages.Source: §6 · p. 9
- Offline and online experiments demonstrate significant improvement over state-of-the-art baseline models.Source: §6 · p. 9
A.1 Reproducibility Information
The appendix reports LightGBM settings and shared neural-network configurations used for reproducibility. Neural-network hyperparameters were tuned on the validation set, and experiments ran on NVIDIA Tesla V100 GPUs.
A.1 Reproducibility Information
- LightGBM uses learning rate 0.1, feature fraction 0.9, bagging fraction 0.7, bagging frequency 5, max bin 2000, and 70 leaves.The settings were chosen by grid search on the validation set; boosting type is gbdt.Source: §A.1 · p. 10
- Neural-network models share common hyperparameters, including embedding dimension 5, Tower output dimension 32, and calibrator strength 0.6.For the industrial dataset, activation negatives were downsampled to make positive samples 1%, except in the test set.Source: §A.1 · p. 10
- The Tower is a three-layer MLP with dimensions [128, 64, 32], while the other listed MLPs are single-layer with dimension 32.Experiments used NVIDIA Tesla V100 GPUs with 16G memory; selected hyperparameters were tuned by validation-set grid search.Source: §A.1 · p. 10
A.2 More Experiments
The appendix describes a case study of transferred-information weights for approval and activation. It uses sampled test examples, grouped or ranked by conversion labels and per-sample logloss, and visualizes learned representations with t-SNE.
A.2 More Experiments
- For Figure 7, 500 positive and 500 negative approval examples are selected, and t-SNE plots original, transferred-plus-original, and AIT-learned information.Positive prediction scores are ranked descending and negative scores ascending.Source: §A.2 · pp. 10–11
A.3 Data Collection and Privacy Protection
The industrial dataset contains Meituan Co-Branded Credit Card banner samples, with timing choices intended to support approval and activation labels. The appendix also states that user-identifying features are excluded.
A.3 Data Collection and Privacy Protection
- The dataset uses banner-exposed samples from Meituan Co-Branded Credit Cards collected over a continuous period.Source: §A.3 · p. 11
- Two-level real-time risk judgment makes approval almost real-time, while activation requires card receipt and subsequent activation.Samples were shown a banner at least 14 days earlier to support activation-label accuracy.Source: §A.3 · p. 11
- The features include context and user statistics but exclude features that locate specific users or involve user privacy.The passage links accurate end-to-end conversion identification with disturbing users as little as possible and improving user experience.Source: §A.3 · p. 11