PaperlayerView exact revision on arXiv

Source-linked AI summary

ESCM$^2$: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation

Hao Wang, Tai-Wei Chang, Tianqiao Liu, Jianmin Huang, Zhichao Chen, Chao Yu, Ruopeng Li, Wei Chu

arXiv:2204.05125v2cs.AIcs.IR

TL;DR

Post-click conversion-rate estimation faces sample selection bias and data sparsity, while ESMM does not guarantee unbiased CVR estimates and may overlook click-to-conversion causality. The paper adds counterfactual risk minimization to ESMM and reports that ESCM2 largely mitigates these issues and performs better than baseline models.

  • Problem

    CVR estimation faces sample selection bias from clicked-only training and data sparsity, while ESMM does not guarantee unbiased CVR estimates.

    Source: §1 · pp. 1–2

  • Method

    ESCM2 uses a counterfactual risk minimizer, including IPS and doubly robust regularization, to regularize ESMM.

    Source: §1 · p. 2

  • Results

    ESCM2 largely mitigates IEB and PIP and achieves better performance than baseline models.

    Source: §7 · p. 10

  • Takeaways & Limitations

    Counterfactual regularization provides a principled way to augment ESMM against CVR estimation bias and CTCVR independence priority.

    Source: §7 · p. 10

    Extending the causal framework to more complex sequential user behaviors remains future work.

    Source: §7 · p. 10

Abstract

from arXiv · show

Accurate estimation of post-click conversion rate is critical for building recommender systems, which has long been confronted with sample selection bias and data sparsity issues. Methods in the Entire Space Multi-task Model (ESMM) family leverage the sequential pattern of user actions, i.e. $impression\rightarrow click \rightarrow conversion$ to address data sparsity issue. However, they still fail to ensure the unbiasedness of CVR estimates. In this paper, we theoretically demonstrate that ESMM suffers from the following two problems: (1) Inherent Estimation Bias (IEB), where the estimated CVR of ESMM is inherently higher than the ground truth; (2) Potential Independence Priority (PIP) for CTCVR estimation, where there is a risk that the ESMM overlooks the causality from click to conversion. To this end, we devise a principled approach named Entire Space Counterfactual Multi-task Modelling (ESCM$^2$), which employs a counterfactual risk miminizer as a regularizer in ESMM to address both IEB and PIP issues simultaneously. Extensive experiments on offline datasets and online environments demonstrate that our proposed ESCM$^2$ can largely mitigate the inherent IEB and PIP issues and achieve better performance than baseline models.

1 INTRODUCTION

CVR estimation is challenged by sample selection bias and sparse clicked data; ESMM addresses sparsity but does not guarantee unbiased estimates. ESCM2 adds counterfactual regularization to ESMM to address inherent estimation bias and potential independence priority.

1 INTRODUCTION

  • Clicked-only training causes sample selection bias, while sparse clicks make naïve CVR models struggle to generalize beyond the click space.Items with lower CVR are less likely to be clicked and therefore included in training.Source: §1 · pp. 1–2
  • ESMM mitigates data sparsity through parameter sharing, but its CVR estimates are not guaranteed to be unbiased.The paper identifies inherent estimation bias (IEB) in CVR estimates and potential independence priority (PIP) in CTCVR estimates.Source: §1 · p. 2
  • ESCM2 uses a counterfactual risk minimizer, including IPS and doubly robust regularization, to regularize ESMM's CVR estimation.The paper states that the CRM regularizer frees CVR and CTCVR estimates from IEB and PIP, respectively.Source: §1 · p. 2
  • The paper reports mathematical proofs and experiments supporting its claims that ESCM2 addresses IEB and PIP.It presents ESCM2 as a causal-perspective improvement to ESMM.Source: §1 · p. 2

2 PRELIMINARIES

The preliminaries define the exposure-space variables and ideal conversion loss, then describe ESMM and propensity-score methods used to address estimation over clicked samples.

2.1 Notations

  • Uppercase letters denote random variables, lowercase letters their values, and calligraphic letters the corresponding sample spaces.The notation section also uses P() for a random variable's probability distribution.Source: §2.1 · p. 2

2.2 Problem Formulation

  • The formulation defines users, items, the exposure space D, the click matrix O, and observed conversion labels R.The ideal loss uses the conversion prediction error, with binary cross entropy specified for that error; because conversions are observed only for clicked pairs, a naïve click-space expectation is biased.Source: §2.2 · p. 2

2.3 Entire Space Multitask Model Approach

  • The paper distinguishes its theoretical proof of ESMM's CVR bias from Zhang et al.'s numerical examples [27].It describes its proof as complete and based on probabilistic tools.Source: §2.3 · p. 2
  • ESMM uses CTR and CVR towers whose product estimates CTCVR, trains CTR and CTCVR over the entire impression space, and uses its CVR tower at inference.The paper says this avoids modeling CVR over the click space but yields inherent CVR overestimation.Source: §2.3 · p. 3

2.4 Propensity Score Based Approach

  • IPS weights click-space error terms by inverse propensity to align their distribution with exposure-space errors, and its unbiasedness requires accurate estimated propensities.The doubly robust estimator adds imputed errors and corrects their deviation for clicked events; unbiasedness holds if either imputation error or propensity score is accurate.Source: §2.4 · p. 3

3 DISCUSSION ON ESMM

The paper identifies an inherent upward bias in ESMM's CVR estimates and a separate risk that its CTCVR model overlooks click-to-conversion causality. It proposes a counterfactual formulation that addresses both issues while avoiding the selection bias of click-only training.

3.1 Is ESMM an Unbiased CVR Estimator?

  • ESMM's CVR estimate is always higher than ground truth, and the paper provides a theoretical proof of this inherent estimation bias.The paper notes that prior researchers had perceived the bias, but a theoretical proof was lacking.Source: §3.1 · p. 3

3.2 Is ESMM a well-defined CTCVR Estimator?

  • ESMM risks modeling conversion independently of clicking for CTCVR, replacing P(R=1|O=1) with P(R=1) and omitting the O→R causal link.The paper calls this potential independence priority (PIP); click-space training restores the link but introduces sample selection bias.Source: §3.2 · p. 4
  • Using P(R=1|do(O=1)) for CVR yields a CTCVR estimand that addresses PIP and selection bias for the full inference space.The intervention removes the dependence between X and O, as depicted in Figure 3(c).Source: §3.2 · p. 4, Figure 3 · p. 4

4 PROPOSED METHOD

ESCM2 regularizes ESMM's CVR estimation with inverse propensity score (IPS) or doubly robust (DR) counterfactual risk minimizers. Its three-task architecture retains CTR and CTCVR objectives while adding counterfactual CVR risk.

4.1 Counterfactual Risk Regularizer

  • ESCM2 uses IPS and DR counterfactual risk minimizers to regularize ESMM and mitigate its inherent estimation bias and potential independence priority.The regularizers target CVR estimation while preserving the multi-task setting.Source: §4.1 · p. 4

4.1.1 Inverse Propensity Score Regularizer.

  • With an accurate CTR estimate, the IPS regularizer is equivalent to the ideal exposure-space loss and encourages unbiased CVR estimates that condition conversion on clicking.The regularizer minimizes counterfactual CVR error across groups; ESMM's CTR output estimates the propensity score.Source: §4.1.1 · pp. 4–5

4.1.2 Doubly Robust Regularizer.

  • DR weights the CVR error deviation using click propensity and is unbiased if either the propensity or imputed error estimate is accurate.An auxiliary imputation loss is introduced to improve the imputed error estimate; DR also mitigates IEB and PIP.Source: §4.1.2 · p. 5, §4.1.1 · p. 5

4.2 Architecture of ESCM2

  • ESCM2 jointly optimizes CTR, CTCVR, and counterfactual CVR risk, sharing embeddings across tasks; ESCM2-DR adds an imputation tower to ESCM2-IPS.The shared feature representations are intended to alleviate data sparsity, since CTR provides substantially more positive feedback than CVR and CTCVR.Source: §4.2 · p. 5, Figure 4 · p. 6

5 EXPERIMENTS

The experiments are organized around comparative performance, bias and independence-priority tests, and sensitivity to objective weights. They use industrial and public data, a shared training protocol, and several ranking metrics.

5 EXPERIMENTS

  • The experiments ask whether ESCM2 outperforms state-of-the-art estimators, reduces ESMM's bias and independence priority, and is sensitive to objective weights.These are framed as four research questions covering performance, bias, PIP, and the effects of λc and λg.Source: §5 · p. 6

5.1.1 Datasets.

  • The industrial data come from 90-day recommender-system logs split chronologically, with training negatives downsampled to an exposure:click:conversion ratio of about 100:10:1.The paper also uses Ali-CCP for reproducible benchmarking and randomly holds out 10% of training data for validation.Source: §5.1.1 · p. 6

5.1.2 Competitors.

  • The competitor set includes naïve and multi-task methods, ESMM, and debiased MTL-EIB, MTL-IPS, and MTL-DR approaches.Single-task IPS and DR methods are excluded for fairness because the evaluation focuses on multi-task learning approaches.Source: §5.1.2 · pp. 6–7

5.1.3 Training Protocol.

  • All models use MMoE and are trained for 300k Adam iterations with shared hyperparameters; λg is 1 and λc is 0.1.The objective weights are set based on the parameter study, and the best validation checkpoint is evaluated on test data.Source: §5.1.3 · p. 7

5.1.4 Evaluation Protocol.

  • AUC is the primary ranking metric; KS, Recall, and F1 are also reported at their respective best ROC or PR thresholds.The paper notes that AUC summarizes ranking across all thresholds.Source: §5.1.4 · p. 7

5.2 Performance Comparison

Offline and online evaluations report stronger performance for ESCM2 than competing methods, with online gains across business and conversion metrics in three scenarios.

5.2.2 Online A/B results.

  • Across three online scenarios, ESCM2 increased order quantity and premium by 2.84% and 10.85%, 4.26% and 3.88%, and 40.55% and 12.69%, respectively.The online experiments used randomized user buckets to compare the models.Source: §5.2.2 · pp. 7–8

5.2.1 Offline results.

  • ESCM2 significantly improved performance over state-of-the-art baselines in offline CVR estimation.Table 2 reports mean and standard deviation over ten runs.Source: §5.2.1 · p. 7, Table 2 · p. 8
  • On CTCVR estimation, ESCM2-IPS led on Ali-CCP F1 and Recall, while ESCM2-DR led on industrial-dataset AUC and KS.ESMM also achieved the highest industrial-dataset AUC and Recall and Ali-CCP F1 and Recall among multi-task baselines.Source: §5.2.1 · p. 7, Table 3 · p. 8

5.3 Discussion on Inherent Estimation Bias

The reported comparison supports the claim that ESMM overestimates CVR, while counterfactual regularization reduces the estimation error.

5.3 Discussion on Inherent Estimation Bias

  • On Ali-CCP, ESMM's CVR expectation was 0.01 versus a ground-truth expectation no higher than 0.0056 on train, and it overestimated test CVR by 0.0057 on average.The table's “Label” is the clicked-space ground-truth conversion rate, treated as an upper bound for the exposure-space expectation.Source: §5.3 · p. 8, Table 5 · p. 8
  • Counterfactual regularization reduced expectation error: IPS by 0.031 on train and 0.035 on test, and DR by 0.037 and 0.045, respectively.Source: §5.3 · p. 8

5.4 Discussion on Potential Independence Priority

The discussion reports evidence of weak click-to-conversion causality in ESMM estimates and examines how counterfactual and global risk weights affect ranking performance.

5.5 Parameter Sensitivity and Ablation Study

  • With λc increased from 0 to 1.5, ESCM2-IPS AUC for CVR rose from 0.755 to approximately 0.78; CTCVR AUC rose from 0.817 at λc = 0 to 0.821 at λc = 0.1.The authors report that large λc can harm CTCVR estimation and generally finetune λc within [0,0.1].Source: Figure 6 · p. 9, §5.5 · p. 9

5.4 Discussion on Potential Independence Priority

  • ESMM's click-to-conversion causal strength was negligible on both datasets, while ESCM2 significantly augmented it by directly modeling the dependency.Strength is measured as the absolute error between the causal risk ratio and 1; Figure 5 reports industrial and Ali-CCP results.Source: §5.4 · p. 9

5.5 Parameter Sensitivity and Ablation Study

  • As λg increased from 0 to 2.5, ESCM2-IPS AUC rose from 0.758 to approximately 0.781, and ESCM2-DR KS rose from 0.385 to approximately 0.434.Within the tested range [0,3], increasing λg consistently improved both CVR and CTCVR estimation.Source: Figure 7 · p. 9, §5.5 · p. 9

6 RELATED WORK

Related work addresses missing-not-at-random feedback through sequential behavior modeling or unbiased estimation on biased data.

6 RELATED WORK

  • Task dependencies make multi-task learning challenging because feedback labels are always missing not at random.Source: §6 · p. 9
  • One line of work uses sequential behavior and probability decomposition, including ESMM, ESM2, GMCM, and HM3.ESMM models CTCVR as CTR multiplied by CVR and learns CVR indirectly.Source: §6 · p. 9
  • A second line builds unbiased estimators on biased datasets, including IPS and doubly robust methods and multi-task debiasing approaches.The cited examples also include a dual-learning framework that eliminates confounding in clicked and unclicked data.Source: §6 · p. 9

7 CONCLUSION AND FUTURE WORK

ESCM2 augments ESMM with counterfactual regularization to mitigate inherent estimation bias and potential independence priority. The authors identify complex sequential behaviors and propensity-score fragility as future directions.

7 CONCLUSION AND FUTURE WORK

  • ESCM2 augments ESMM with counterfactual regularization and experiments show it largely mitigates IEB and PIP while outperforming baseline models.The conclusion presents ESCM2 as a principled approach to the two issues.Source: §7 · p. 10
  • Future work will extend the causal framework to more complex sequential user behaviors, potentially using cascaded counterfactual regularization.The authors note that probabilistic task decomposition in existing approaches, as in ESMM, can introduce IEB and PIP.Source: §7 · p. 10
  • A second future direction is addressing propensity-score fragility, since IPS/DR unbiasedness heavily relies on accurate CTR estimates.The authors suggest replacing weighting-based distribution alignment with adversarial training or representation learning.Source: §7 · p. 10

A IMPLEMENTATION DETAILS

ESCM2-IPS builds on ESMM, estimates propensity scores, computes a clicked-space CVR loss, and optimizes the resulting objective. Implementation safeguards address unstable propensity weights and protect CTR estimation.

A IMPLEMENTATION DETAILS

  • ESCM2-IPS starts with ESMM's CTR and entire-space CTCVR losses, then uses the CTR estimate as a shortcut propensity score.The procedure calculates the CVR estimation error in clicked space using δ and the propensity score.Source: A IMPLEMENTATION DETAILS · p. 10
  • The objective is optimized with stochastic gradient methods, and small propensity scores are clipped to limit estimation variance and numerical error.The implementation uses a threshold such as 0.1.Source: A IMPLEMENTATION DETAILS · p. 10
  • The CVR-loss gradient to the CTR estimate is truncated to avoid biasing CTR estimation.The authors also advise prioritizing accurate CTR estimates because they significantly affect CVR and CTCVR estimates.Source: A IMPLEMENTATION DETAILS · p. 10
  • ESCM2-DR follows the same procedure but calculates the CVR loss with Equation 22, and alternating training can stabilize its DR regularizer.The implementation also suggests a small λc weight (0.1–1) and a larger CTR tower to support accurate CTR estimation.Source: A IMPLEMENTATION DETAILS · p. 10
Loading 2204.05125v2…