PaperlayerView exact revision on arXiv

Source-linked AI summary

Doubly Robust Estimation of Causal Effect on CVR with Targeted Regularization

Jiayi Dan, Bo Li, Lu Deng, Yong Wang

arXiv:2608.13461v1cs.LG

TL;DR

Stage-wise conversion effects have received less attention than overall conversion probability, although CVR describes second-stage conversion. This paper uses semiparametric theory to construct a doubly robust CVR estimator with targeted regularization; it reports √n-consistency under mild conditions and experimental advantages over indirect loss-debiasing.

  • Problem

    Causal estimation of stage-wise conversion effects has received less attention than overall conversion probability, despite CVR's role as a metric of second-stage conversion.

    Source: §1 · p. 1

  • Method

    Using semiparametric theory, the paper derives an influence function and von Mises expansion to construct a doubly robust CVR estimator, then adds targeted regularization for stability.

    Source: §1 · p. 2

  • Results

    √n-consistency under mild conditions is reported for the estimator, and experiments demonstrate strong performance and advantages over indirect loss-debiasing.

    Source: §7 · p. 10

  • Takeaways & Limitations

    Reported advantages over indirect loss-debiasing support tailoring estimation to the CVR target rather than relying on an indirect debiasing approach.

    Source: §1 · p. 2, §7 · p. 10

    With flexible nonparametric models such as neural networks, nuisance estimators may fail to achieve √n convergence.

    Source: §4.1 · p. 4

Abstract

from arXiv · show

Post-click conversion rate (CVR) is a key metric in various scenarios including e-commerce and advertising, reflecting the efficiency and user experience in the second stage of the conversion process. Estimating the causal effect on CVR is therefore of great practical importance. However, directly applying existing causal inference methods to clicked samples introduces sample selection bias and increased variance due to the exclusion of non-click data. Recent studies on CVR prediction introduce "ideal loss", which optimizes model parameters using an unbiased estimate of the loss over the full sample. Nevertheless, there is no guarantee that unbiasedness of the loss implies unbiasedness of the final estimator. We revisit this challenge from the perspective of semiparametric theory. Specifically, we develop a new doubly robust causal effect estimator for chain-structured outcomes such as CVR, and derive its theoretical properties in detail. It achieves a faster convergence rate compared to nuisance parameters estimation and is therefore more robust when using flexible nonparametric estimators, including neural networks. Based on these theoretical findings, we further design a framework based on targeted regularization to improve numerical stability and practical applicability. Extensive experiments on synthetic and real-world data demonstrate the effectiveness and robustness of our method. In addition, we find that naively combining loss debiasing with standard causal estimators underperforms our method, highlighting the necessity of developing the new estimator tailored to this CVR-style objective with solid theoretical guarantees.

1. Introduction

The paper develops a semiparametric doubly robust estimator tailored to CVR causal effects, addressing click-selection bias and the lack of guarantees that unbiased loss yields an unbiased estimand. It also proposes targeted regularization and evaluates the method on synthetic, semi-synthetic, and real-world data.

1. Introduction

  • CVR is P(Y2 = 1 | Y1 = 1), a second-stage efficiency metric that complements overall conversion by identifying policies that may improve clicks while reducing post-click conversion.Source: §1 · p. 1
  • Mediation and controlled direct effect methods are unsuitable because their identifiability assumptions may fail and clicked-sample regression is biased under the structural dependence of Y2 on Y1.Source: §1 · pp. 1–2
  • Estimating CVR effects only among clicked users can bias full-population conclusions; Huang et al. (2024) address this only for RCT data, while ideal-loss methods lack guarantees that unbiased loss yields an unbiased estimand.The ideal-loss methods cited are Wang et al. (2019), Zhang et al. (2020), Guo et al. (2021), Dai et al. (2022), and Li et al. (2022b;a).Source: §1 · p. 2
  • The proposed semiparametric doubly robust estimator for chain-structured outcomes such as CVR remains consistent if one nuisance estimator is inconsistent and has desirable theoretical properties.Its construction uses the influence function and von Mises expansion of the target estimand.Source: §1 · p. 2
  • Targeted regularization builds on the theoretical results to improve the practical applicability, stability, and empirical performance of the estimation framework.Source: §1 · p. 2
  • The authors report strong performance and robustness across synthetic, semi-synthetic, and real-world data experiments.Source: §1 · p. 2

2. Related Works

Prior work addresses causal estimation with deep learning and doubly robust methods, but existing CVR approaches do not directly target the final causal estimand with theoretical guarantees and practical utility.

2.1. Causal Inference based on Deep Learning

  • Deep-learning causal methods use balanced representations, propensity scores, weighting, and specialized architectures, but applying them to CVR on clicked samples risks selection bias and data sparsity.Prior work includes generalization bounds using empirical risk and IPM distance, extensions to continuous treatment, and VAE, GAN, and diffusion architectures.Source: §2.1 · p. 2

2.2. Doubly Robust Estimator and Targeted Regularization

  • Doubly robust estimators combine bias correction with √n convergence under appropriate conditions, and targeted regularization addresses finite-sample instability through a perturbation parameter.R-Learner, asymptotic and efficiency analyses, and continuous-treatment extensions build on this foundation, but CVR requires adapting to different estimands and nuisance parameters.Source: §2.2 · pp. 2–3

2.3. Debiased CVR Prediction

  • Ideal-loss methods debias CVR prediction losses rather than the final causal estimand, while Huang et al. (2024) propose a randomized-data-only plug-in approach lacking double robustness and consistency.The literature therefore leaves few methods that directly target CVR causal effects while providing sound theoretical guarantees and practical utility.Source: §2.3 · p. 3

3. Problem Formulation and Notations

The formulation models conversion as a click-dependent outcome under treatment, with covariates and treatment potentially associated. Restricting estimation to clicked samples is biased because clicking is non-random.

  • The data are i.i.d. observations of covariates, treatment, click, and conversion, with conversion implying a click; treatment is modeled as continuous, extending naturally to discrete treatment (Nie et al., 2021).Source: §3 · p. 3
  • Restricting estimation to clicked samples introduces bias because click status is non-random, while treatment affects both click and conversion.Source: §3 · p. 3
  • The estimator uses three nuisance parameters: conditional click probability µ1(x, a), conditional conversion probability µ2(x, a), and treatment density π(a | x).Source: §3 · p. 3
  • The analysis assumes overlap, no unblocked back-door paths from treatment to either outcome given covariates, and a positive lower bound on click probability µ1.The text motivates the lower bound by finite populations and notes that cases with µ1 = 0 have undefined CVR and are outside scope.Source: §3 · p. 3

4. Doubly Robust Estimation

The section contrasts a plug-in CVR estimator whose consistency depends on nuisance estimation with a doubly robust estimator that removes first-order bias and is root-n consistent under mild assumptions. Because direct one-step correction can be unstable, the authors extend targeted regularization to their estimand.

4.1. Plug-in Estimator and Nuisance Parameter Estimation

  • The plug-in estimator takes the ratio of predicted μ2(x, a) and μ1(x, a), but its convergence depends on the slowest nuisance estimator and can fail under flexible nonparametric estimation.The method also predicts the treatment propensity to inform nuisance estimation; for continuous treatment, it uses a varying coefficient model to preserve continuity and smoothness.Source: §4.1 · p. 4

4.2. The New Doubly Robust Estimator

  • The new estimator subtracts an estimated first-order bias term using a one-step correction, leaving error dominated by second-order nuisance-estimation errors.The correction is based on the influence function of the target estimand and the distributional Taylor expansion.Source: §4.2 · pp. 4–5
  • Under mild assumptions, the doubly robust estimator is root-n consistent and asymptotically normal, even when flexible nuisance estimators do not achieve root-n convergence.Cross-fitting can satisfy Condition 1, while neural networks can meet the oP(n^-1/4) nuisance-rate conditions.Source: §4.2 · p. 5
  • Direct one-step correction may be unstable or yield estimates outside the parameter space, especially because low click-through rates can make its μ1 denominator small.The required cross-fitting also adds complexity, motivating a targeted-regularization framework for this estimand.Source: §4.2 · p. 5

5. A Practical Framework Based on Targeted Regularization

The section adapts targeted regularization to the CVR estimand, where the established theoretical foundation for standard causal estimands is unavailable. It presents an end-to-end framework with doubly robust properties and a multi-task extension for jointly estimating CTR and CVR effects.

5. A Practical Framework Based on Targeted Regularization

  • The framework learns an additional parameter to approximate the correction term and combines the plug-in loss with a targeted-regularization penalty.The penalty is weighted by hyperparameter β.Source: §5 · p. 6
  • Under mild regularity conditions, the estimator remains doubly robust, converges faster than individual nuisance estimators, and supports flexible nonparametric models, including neural networks.The convergence rate is dominated by the remainder after targeted regularization controls the first-order bias.Source: §5 · p. 6
  • Targeted regularization softens the original estimator’s correction, can improve stability without affecting nuisance-estimation consistency, and enables end-to-end estimation without cross-fitting.Stability can be further enhanced by controlling the learning rate of ϵ or adding regularization on ϵ.Source: §5 · p. 6
  • A multi-task model jointly estimates causal effects on CTR and CVR, with µ2 predicted as µ1 multiplied by a factor constrained to (0, 1).Gradients from the targeted-regularization terms are blocked from the propensity-score estimate to improve its accuracy.Source: §5 · pp. 6–7

6. Experiments

Across synthetic, semi-synthetic, and supplementary real-world experiments, the proposed method outperforms baselines on CVR estimation; targeted regularization contributes to performance, and direct loss debiasing performs worse than the tailored estimator.

6.1.2. BASELINES AND SETTINGS

  • Experiments compare DragonNet, VCNet, DRNet, TARNet, Causal Forest, ECUP, a cross-fit doubly robust estimator, and a no-targeted-regularization ablation using unified model capacity.Source: §6.1.2 · pp. 7–8

6.1.1. SYNTHETIC AND SEMI-SYNTHETIC DATA GENERATION

  • AMSE is the primary evaluation metric, and the synthetic and News-based semi-synthetic data are constructed to provide known target values.Synthetic datasets have 3,000 training and 1,000 test samples; the semi-synthetic datasets reuse 498-dimensional News covariates. Observed outcomes have mean Y1 below 0.5 and P(Y2 = 1 | Y1 = 1) below P(Y1 = 1), consistent with real-world scenarios.Source: §6.1.1 · p. 7, Table 1 · p. 7, §6.1.3 · p. 8

6.2.1. OVERALL PERFORMANCE AND ABLATION STUDY

  • Removing targeted regularization significantly worsens performance; the basic doubly robust estimator is competitive but inferior to the final method.Source: §6.2.1 · p. 8

6.2.2. HYPERPARAMETER SENSITIVITY ANALYSIS

  • Performance is stable across a wide range of propensity-loss weights α, while AMSE degrades monotonically as targeted-regularization weight β decreases.Across α settings, AMSE remains substantially lower than in the ablation study.Source: Figure 2 · p. 8, Figure 3 · p. 8, §6.2.2 · p. 8

6.2.1. OVERALL PERFORMANCE AND ABLATION STUDY

  • Our method significantly outperforms all baselines on the CVR task across datasets, while CTR performance remains desirable with added task and regularization terms.The results support jointly estimating causal effects on CTR and CVR.Source: §6.2.1 · p. 8, Table 2 · p. 9

6.2.3. COMPARISON WITH DIRECT LOSS DEBIASING

  • The alternative IPS loss-debiasing approach beats the plug-in estimator without targeted regularization and most Table 2 baselines, but performs worse than our proposed method.It debiases loss terms rather than directly targeting the final estimand, for which unbiased loss estimation does not guarantee accurate estimation.Source: §6.2.3 · p. 9, Table 3 · p. 10

6.3. Performance on Real-World Dataset

  • On CRITEO-UPLIFTv2, our method significantly outperforms other models for CVR causal-effect estimation, and the ablation again shows targeted regularization improves performance.The authors caution that this real-world evaluation is supplementary: the dataset has binary treatment, lacks the true estimand, and uses individual-level AUUC and QINI despite average-level theory.Source: §6.3 · pp. 9–10

7. Conclusion

The paper introduces a doubly robust estimator tailored to CVR causal effects with √n-consistency under mild conditions, and a targeted-regularization framework for practical use. Experiments support the method’s performance and the effectiveness of targeted regularization.

  • The proposed doubly robust estimator is tailored to the CVR causal-effect estimand and achieves √n-consistency under mild conditions.Source: §7 · p. 10
  • The authors develop a practical framework based on targeted regularization, building on the estimator’s theoretical findings.Source: §7 · p. 10
  • Experiments demonstrate the method’s strong performance, targeted regularization’s effectiveness, and advantages over indirect loss debiasing.Source: §7 · p. 10

8. Impact Statement

The paper presents a general framework for estimating causal effects of chain-structured outcomes, with potential applications in e-commerce and online advertising decision-making.

8. Impact Statement

  • The framework estimates causal effects for chain-structured outcomes such as CVR.It is proposed as a general framework.Source: §8 · p. 10
  • The method can be applied to decision-making in e-commerce and online advertising.Source: §8 · p. 10

A. Appendix · A.1. Proof of Lemma 4.1

The proof assumes positive lower bounds for μ1 and μ̂1 so that f(x, y) = y/x is twice continuously differentiable, then applies a second-order Taylor expansion with a Lagrange remainder.

A.1. Proof of Lemma 4.1

  • The supplied proof fragment includes the density expression p(y1, x, a) integrated over y1 and x.Source: §A.1 · p. 13
  • The supplied proof fragment also includes p(y1 | x, a) integrated over y1 and dP(x).Source: §A.1 · p. 13
  • Positive lower bounds for μ1 and μ̂1 make f(x, y) = y/x twice continuously differentiable on the specified region.This condition supports the Taylor expansion used in the proof.Source: §A.1 · p. 13
  • The proof applies a second-order Taylor expansion of a bivariate function with a Lagrange remainder.Source: §A.1 · p. 13
  • The proof introduces a remainder term Rμ and states that it satisfies a condition whose displayed expression is not present in the supplied passage.Source: §A.1 · p. 13
  • A bounded constant C is introduced before the proof proceeds to its next step.Source: §A.1 · p. 13

A.2. Proof of Theorem 4.2 · A.3. Proof of Theorem 5.1 · A.4. Synthetic and Semi-synthetic Data Generation

The appendices justify asymptotic normality and a spline-based rate bound, then describe synthetic and semi-synthetic data-generation setups.

A.2. Proof of Theorem 4.2

  • The proof establishes asymptotic normality by combining a CLT leading term with o_P(n^-1/2) remainder control and Assumption 1 via Slutsky’s theorem.Source: §A.2 · p. 13
  • With probability-one lower bounds on the estimated propensity and conversion regression, the proof obtains R2(ˆP, P) = oP(n^-1/2).Source: §A.2 · p. 13

A.3. Proof of Theorem 5.1

  • The proof decomposes its bound into T1, T2, and T3, with displayed rate terms log n + r1(n)r2(n) + r1(n)r3(n).It uses a spline-function argument from (Nie et al., 2021), which is independent of the target estimand.Source: §A.3 · pp. 13–14

A.4. Synthetic and Semi-synthetic Data Generation

  • Synthetic covariates are drawn uniformly from [0, 1]^8, and the component forms follow related work (Nie et al., 2021).Source: §A.4 · p. 14
  • The synthetic outcomes encode click and conversion as binary variables, with conversion set to zero whenever no click occurs.Source: §A.4 · p. 14
  • For semi-synthetic data, four parameters are formed by normalizing vectors sampled from a standard normal distribution.Source: §A.4 · p. 14

Related papers

Loading 2608.13461v1…