Enhanced Doubly Robust Learning for Debiasing Post-click Conversion Rate Estimation

1 INTRODUCTION

• Post-click conversion rate estimation matters because conversion feedback expresses user preference and contributes directly to platform GMV.Conversion feedback is observed only for clicked events, which comprise a small fraction of possible user behaviors.Source: §1 · p. 1
• Clicked events are not representative of all events because users self-select items they are more likely to purchase, making missing conversions MNAR.The paper therefore adopts an MNAR assumption and focuses on selection bias.Source: §1 · p. 1
• Model-agnostic imputation may not track the prediction model’s changing gradient target, while inaccurate imputation can raise DR variance above IPS.The paper identifies imputation learning as an under-investigated component of DR estimation.Source: §1 · p. 2
• The proposed MRDR estimator reduces DR variance while retaining double robustness, and double learning makes imputation dynamically model-correlated.The approach uses two CVR models and periodically updates the imputation model from the prediction model.Source: §1 · p. 2
• MRDR-DL outperforms state-of-the-art methods in experiments on a semi-synthetic dataset and two real-world datasets.The authors also report that MRDR achieves the best relative error across all semi-synthetic settings.Source: §1 · p. 2

2.1 Problem Formulation

• Because conversions are observed only for clicked events, the naive loss estimate is biased under MNAR feedback, motivating unbiased estimators.The ideal loss is defined over all user–item pairs, while the naive estimate averages errors for clicked events.Source: §2.1 · pp. 2–3

2.2 Error Imputation Based Estimator

• EIB estimates errors for unclicked events with an imputation model, but inaccurate imputations can produce large bias and mislead prediction-model learning.EIB is unbiased when imputed errors are accurate for every unclicked event.Source: §2.2 · p. 3

2.3 Inverse Propensity Score Estimator

• IPS weights clicked-event errors by inverse propensity and is unbiased when estimated propensities are accurate, but typically suffers especially high variance.The propensity is the probability that an item is clicked by a user.Source: §2.3 · p. 3

2.4 Doubly Robust Estimator

• DR combines imputed errors with propensity-weighted error corrections and is unbiased if either the imputation or clicked-event propensity estimates are accurate.Inverse-propensity weighting also creates a high-variance problem for imputation learning.Source: §2.4 · p. 3

3.1 Bias and Variance Analysis of DR Estimator

• DR bias is near zero when either the imputation error or propensity deviation is near zero, unlike EIB and IPS, which each require one to be near zero.This is the estimator’s double robustness.Source: §3.1 · p. 3

3.2 More Robust Doubly Robust Estimator

• Under inaccurate error imputation, DR can have higher variance than IPS; MRDR learns imputation by minimizing DR variance while retaining double robustness.MRDR’s imputation loss changes the weights from 1/ˆp_u,i to (1 − ˆp_u,i)/ˆp_u,i^2, increasing penalties for clicked events with low propensity and decreasing them for other clicked events.Source: §3.2 · p. 4

3.3 Double Learning Approach

• The double-learning approach uses pseudo-labels to make imputed errors dynamically model-correlated, recasting error imputation as general CVR estimation.The prior model-agnostic imputation model estimates gradients for prediction, but does not dynamically track the prediction model.Source: §3.3 · p. 5
• The two models have the same structure but different parameters; cross-entropy between pseudo-labels and predicted CVR trains imputation as a binary classification task.The approach alternates model learning, mutually regularizes the models, and periodically copies prediction-model parameters to the imputation model to address high imputation variance.Source: §3.3 · p. 5

1 Initialize the parameters 휃, 휙

• For imputation training, sample batches of clicked events and update the imputation parameters by descending along the gradient.Source: §1 · p. 5
• For prediction training, sample batches from D1 and update the prediction parameters by descending along the gradient.Source: §1 · p. 5
• Before training the imputation model, copy the prediction-model parameters to it to periodically correct the imputation model and address its high variance.The paper reports this scheme as empirically beneficial for eliminating the high-variance problem of imputation learning.Source: §1 · p. 5

4.1.1 Dataset and preprocessing.

• Because real-world datasets lack fully observed conversion labels, the study constructs a semi-synthetic evaluation dataset from MovieLens 100K.The procedure converts explicit feedback into fully observed conversion-label and click-indicator matrices.Source: §4.1.1 · pp. 5–6
• The preprocessing completes ratings with matrix factorization, transforms predicted ratings into CTR and assigned CVR values, then samples clicks and conversions.The assigned CVR values are 0.1, 0.3, 0.5, 0.7, and 0.9 for ratings 1 through 5.Source: §4.1.1 · p. 6

4.1.2 Experimental details.

• The experiments compare five estimators on five predicted-CVR matrices using relative error; lower mean relative error indicates more accurate loss estimation.The compared prediction matrices include ONE, THREE, FIVE, SKEW, and CRS.Source: §4.1.2 · p. 6

4.2 Experiment Results (RQ1)

• MRDR achieves the best relative error across all settings, while IPS, DR, and MRDR outperform the naive estimator and EIB is worst in four settings.The authors attribute MRDR’s performance to double robustness and reduced variance; Table 2 reports averaged relative error over 20 sampling runs.Source: Table 2 · p. 6, §4.2 · p. 6

5 REAL-WORLD EXPERIMENTS

• The real-world experiments ask whether MRDR-DL improves debiasing, how design choices and unclicked-event sampling affect performance, and how double learning works for both models.Source: §4.1.3 · p. 6

5.1.1 Datasets and preprocessing.

• Only Coat Shopping and Yahoo! R3 are identified as publicly available datasets meeting the requirements of explicit feedback and an additional MAR test set.The MNAR data are split into 90% training and 10% validation, while MAR data are reserved for testing.Source: §5.1.1 · p. 7

5.1.2 Baselines.

• The baselines are Naive, IPS, and DR-JL; EIB is excluded because it is considered a weak baseline due to high bias.Both CTR and CVR are estimated with factorization machines.Source: §5.1.2 · p. 7

5.1.3 Experimental Protocols.

• All methods use mini-batch Adam with learning rate 0.001 and batch size 1024; hyperparameters are selected on validation data before early stopping.The protocol reports the best test result from the model selected by validation performance.Source: §5.1.3 · p. 7

4.1.3 Evaluation metric.

• Debiasing performance is evaluated using average per-user recall and discounted cumulative gain (DCG) on the test set.Source: §5.1.3 · p. 7

5.2 Overall Performance (RQ2)

• MRDR-DL consistently outperforms the other methods on both real-world datasets across DCG@K and Recall@K for K ∈ {2, 4, 6}.The authors attribute this to the effectiveness of both the MRDR estimator and double learning approach.Source: §5.2 · p. 7

5.3 Ablation Study (RQ3)

• Replacing MRDR with DR in DR-DL indicates that propensity-adjusted weights for clicked events help reduce DR variance and improve prediction performance.The variant changes imputation-learning weights from (1−p̂_u,i)/p̂_u,i^2 to 1/p̂_u,i.Source: §5.3 · p. 8
• MRDR-JL’s results support periodically correcting the imputation model using the prediction model to address high variance in imputation learning.MRDR-JL alternates model training but skips periodic parameter sharing.Source: §5.3 · p. 8
• With squared loss (SL), the variant matches MRDR-DL on Coat Shopping and significantly outperforms it on Yahoo! R3, though the authors favor cross entropy for pseudo-label classification.Squared loss targets deviation between imputed and true prediction errors; pseudo-label generation is a binary classification problem.Source: §5.3 · p. 8

5.4 Parameter Sensitivity Study (RQ4)

• The best sampling ratio is around 4 to 8 unclicked events per clicked event on both datasets; ratios that are too low or high may hurt prediction performance.Sampling only clicked events performs worst in most settings, while sampling all events also adversely affects performance.Source: §5.4 · p. 8

5.5 Analysis of the Double Learning Approach (RQ5)

• On Coat Shopping, the prediction model’s training loss fluctuates initially, while the imputation model’s curve is more stable; both models converge together after about 900 epochs.DCG@4 evaluates prediction and MAE evaluates imputed errors; the authors relate testing-curve fluctuation to point-wise training loss versus list-wise evaluation.Source: §5.5 · p. 9, Figure 3 · p. 9

6.1 General Approaches to CVR Estimation

• CVR estimation has used traditional, deep-learning, and reinforcement-learning models, but relatively few studies directly address the task and selection bias remains underexplored.Examples include logistic regression, gradient boosting, neural networks, and graph convolution networks,, [18] [32], and.Source: §6.1 · p. 9

6.2 Counterfactual Learning from MNAR Data

• Counterfactual MNAR methods include EIB, IPS, and DR, alongside information-bottleneck, meta-learning, and causal-embedding approaches; specifying imputation and propensity models remains critical.IPS and DR are described as widely applied in recommender systems.Source: §6.2 · p. 9

6.3 Selection Bias in CVR Estimation

• Related CVR debiasing methods include heuristic multi-task approaches and IPS/DR estimators, but prior approaches were evaluated on biased datasets; this paper evaluates learning debiasing using two real-world datasets with unbiased data.The paper focuses on debiasing CVR estimation learning, unlike work on debiasing ranking-metric evaluation.Source: §6.3 · p. 9

• MRDR reduces the DR estimator’s variance while retaining double robustness, and double learning dynamically uses prediction-model information for imputation.The authors report that this learning approach empirically eliminates the high-variance problem of imputation learning.Source: §7 · p. 9
• The proposed approach demonstrates effectiveness on semi-synthetic datasets and superiority on two real-world datasets.Source: §7 · pp. 9–10
• The explainability of the debiasing approach warrants further investigation.Source: §7 · p. 10