Source-linked AI summary
DCEO: Direct Causal Effect Optimization for Long-Term User Value Modeling in E-commerce Search
Junzhao Zhang, Tao Zhang, Liren Yu, Feiyi Dong, Zhixuan Zhang, Dan Ou, Haihong Tang
TL;DR
DCEO addresses the gap between user-level long-term objectives and item-level ranking scores in e-commerce search. It learns context-dependent proxy scores through an actor-critic framework that optimizes relative causal effect, outperforming the conventional GMV proxy by 0.36% in GMV in a 41-day online A/B test.
Problem
User-level long-term objectives cannot directly serve as item-level training labels because items exposed to the same user contribute differently to the outcome.
Method
DCEO aggregates item-level proxy scores into a calibrated user-level metric and uses an actor-critic framework to optimize its relative causal effect on the ultimate objective.
Results
0.36% GMV increase over the conventional GMV proxy in a 41-day online A/B test.
Takeaways & Limitations
DCEO provides context-dependent item-level proxy scores and was effective in a large-scale industrial e-commerce search deployment.
Takeaways & Limitations
The causal interpretation of the relative causal effect requires measured user features to capture major confounders, sufficient local data support, and accurate critic predictions.
Abstract
from arXiv · showhide
Industrial e-commerce search systems ultimately aim to optimize the user-level long-term objective, such as n-day cumulative purchases or gross merchandise value (GMV) per user. However, such objectives are defined at the user level, whereas search ranking is based on item-level scores within each request. Existing methods typically bridge this granularity gap through manually designed multi-objective fusion, where predictions of multiple item-level objectives, such as clicks, carts, purchases, and transaction value, are combined into a ranking score that serves as a proxy for the ultimate objective. Such hand-crafted fusion schemes rely on a small set of manually tuned weights, limiting fine-grained personalization and leading to suboptimal alignment with the ultimate objective. In this paper, we propose DCEO (Direct Causal Effect Optimization), a data-driven framework for learning item-level proxy scores that are better aligned with the ultimate objective. We first aggregate the item-level proxy scores into a user-level proxy metric and quantify its alignment with the ultimate objective using a relative causal effect. We then develop an actor-critic framework, where the critic estimates the ultimate objective for a given user-level proxy metric, and the actor dynamically generates context-dependent fusion weights over multiple objectives to construct the item-level proxy scores and is trained to directly optimize the relative causal effect. Extensive offline experiments and analyses demonstrate the effectiveness and interpretability of DCEO. In addition, DCEO has been deployed in a large-scale industrial e-commerce search system, outperforming the conventional GMV proxy by 0.36% in GMV in a 41-day online A/B test.
1 Introduction
DCEO addresses the mismatch between user-level long-term objectives and item-level ranking by learning personalized proxy scores and optimizing their relative causal effect. It extends existing fusion with a data-driven actor-critic framework, achieving a 0.36% GMV improvement over the conventional GMV proxy in a 41-day online A/B test.
- User-level cumulative purchases or GMV summarize outcomes across requests, while ranking assigns scores to individual items, preventing direct use of the ultimate objective as an item-level label.
- Manual multi-objective fusion combines click, cart, purchase, and transaction-value predictions using globally shared parameters tuned through repeated online A/B tests.The supplied passage truncates the second limitation of this process.
- DCEO uses an actor-critic framework to optimize the relative causal effect of an aggregated user-level proxy metric on the ultimate objective.The actor generates context-dependent weights over selected upstream scores, while the critic estimates the ultimate objective.
- DCEO separates offline training from online serving: only the actor is deployed, and its item-level proxy score is added to the existing fusion formula.
- 0.36% GMV improvement over the conventional GMV proxy was observed in a 41-day online A/B test.The contribution summary reports this result for a large-scale industrial e-commerce search system.
2 Related Work
Related work addresses long-term value through behavior, human feedback, future-value decomposition, trainable fusion, and causal page optimization. DCEO differs by learning item-level proxy scores directly from user-level objectives through estimated relative causal effects.
- Long-Term User Value Modeling: Long-term user-value methods use behavior logs, human feedback, retention rewards, or future-value allocation to connect ranking decisions with delayed outcomes.
- Long-Term User Value Modeling: IURO connects item representations with user-level retention outcomes, but its training-time attention weights are unavailable in the same form during online serving.
- Long-Term User Value Modeling: These long-term modeling methods primarily learn predictive associations, which do not necessarily imply that increasing a signal through ranking improves the ultimate objective.
- Multi-Objective Fusion: Context-dependent fusion methods increase capacity and personalization, but they are trained primarily with item-level behavior labels and task-specific metrics.
- Causal Page Optimization: Whole-page optimization estimates page-level causal effects for fixed whole-page weights, whereas DCEO uses user-level ultimate objectives to learn personalized item-level proxy scores.
3 Problem Formulation
The formulation represents long-term user value at user granularity, constructs item-level proxy scores from existing objectives, calibrates their aggregate to a fixed impression count, and evaluates alignment causally.
- 3.1 User-Level Ultimate Objective and Granularity Gap: For each user, the framework considers all item impressions from requests on a reference day and defines the user-level objective over an n-day outcome window.
- 3.1 User-Level Ultimate Objective and Granularity Gap: Because ranking orders items within requests but the objective summarizes multiple requests, all exposed items would otherwise share one label despite differing contributions.
- 3.2 Multi-Objective Fusion: The ranking stage supplies predicted click, cart, purchase, transaction-value, and other objective scores, which the fusion module combines into a descending-order ranking score.
- 3.3 Item-Level Proxy Score and User-Level Proxy Metric: The proxy score is added to the existing fusion formula so ranking increases the corresponding proxy action rate, but daily counts and rates remain confounded by impression count.
- 3.3 Item-Level Proxy Score and User-Level Proxy Metric: The user-level proxy metric is the fixed-impression action rate, equivalently defined as the expected item-level proxy score at reference impression count C.
- 3.4 Relative Causal Effect of the Proxy Metric on the Ultimate Objective: A relative increase δ in the proxy metric is treated as an intervention, and its effect is evaluated through the resulting relative change in the ultimate objective.
- 3.4 Relative Causal Effect of the Proxy Metric on the Ultimate Objective: A larger positive RCE indicates better alignment between the proxy metric and ultimate objective, so the learning objective maximizes RCE.
4 Method
DCEO trains on user-day impressions and connects context-dependent item-level proxy scores to user-level outcomes through calibrated aggregation and a critic. Online, only the actor computes the added ranking score.
- Training Data: Each training sample contains a user’s reference-day impressions, user and request features, upstream predicted scores, and a user-level ultimate-objective label.
- Critic and Losses: The critic estimates the ultimate objective from user features and the proxy metric, while causal-effect and normalized-ranking losses train the actor.
- Online Serving: During online serving, only the actor is deployed; its proxy score is added to the existing multi-objective fusion score for item ranking.
- Actor: The actor generates request-specific weights from user and request features and combines them with selected upstream scores to produce item-level proxy scores.
- User-Level Proxy Metric: DCEO aggregates item-level proxy scores over impressions and calibrates the aggregate to a reference impression count, producing a user-level proxy metric.The calibration model estimates the average proxy score at the reference count.
4.4 Critic
The critic estimates a user’s ultimate objective from user features and the user-level proxy metric.
- The critic model h_φ predicts the ultimate objective Y_u from user features and proxy metric P_u(θ).It is trained using mean squared error loss.
4.5 Optimization
DCEO trains the actor with causal-effect optimization and conditional normalized ranking regularization, using critic-based estimates to improve proxy alignment while stabilizing training.
- The actor combines causal effect loss LCE with conditional normalized ranking loss LCNR.The two losses jointly shape proxy optimization and regularization.
- LCE estimates the effect of increasing P_u(θ) by a relative amount and uses the negative response difference as the actor loss.The critic is evaluated at P_u(θ) and (1 + δ)P_u(θ).
- The critic parameters are frozen when calculating LCE, so the loss updates only the actor parameters.
- Minimizing LCE approximately maximizes RCE while avoiding ratio-denominator gradient instability.The baseline response varies little across batches, making the unnormalized response difference a stable optimization target.
- LCNR maps proxy–objective pairs toward a common distribution and uses pairwise ordering to provide a coarse, stable approximation to LCE.It estimates conditional means and log standard deviations before applying Bradley–Terry ranking.
- The final actor is trained jointly with the calibration, critic, and normalization models using an overall loss.
4.6 Online Serving
During online serving, DCEO deploys only the actor to generate request-specific weights and item-level proxy scores from available features and upstream predictions.
- Only the actor model f_θ is deployed online to compute request-specific weights.
- The serving system combines selected upstream predicted scores with the actor’s weights to calculate item-level proxy scores.
5 Experiments
Experiments evaluate DCEO offline across objectives, losses, score sets, and aggregation windows, then compare it with a conventional GMV proxy online. DCEO achieves RCE 0.053 offline and increases online GMV by 0.36%.
- Experimental Setup: Offline experiments use 14 consecutive training days followed by one evaluation day, with four-day cumulative GMV as the default objective.
- Experimental Setup: DCEO uses 17 selected upstream predicted scores and fixed final settings C=100, δ=0.05, and α=0.3.
- Experimental Setup: The models are multilayer perceptrons, with the actor producing nonnegative weights from user and request feature embeddings.
- Evaluation Metrics: Relative causal effect is the primary offline metric, and RCE 0.053 represents a 5.3% relative objective increase under a 5% proxy increase.
- Final Model: DCEO achieves RCE 0.053, with impr2click receiving the largest mean weight and several click-to-purchase thresholds receiving substantial weight.
- Final Model: Nonzero weight standard deviations show that DCEO learns context-dependent rather than fixed score combinations.
- Loss Analysis: Replacing predictive association with LCE increases RCE from 0.022 to 0.031, while the complete actor loss is evaluated against both alternatives.
- Loss Analysis: Combining LCE and LCNR yields the best RCE of 0.053 at α=0.3; LCE alone achieves 0.031 and LCNR alone 0.048.
6 Limitations
DCEO’s limitations concern causal identification, incomplete offline effect coverage, restricted actor expressiveness, and limited evaluation scope. These boundaries qualify how its offline RCE and generalizability should be interpreted.
- Causal identification: RCE is a model-based local effect estimate whose causal interpretation depends on observed confounders, sufficient support, and accurate critic predictions.Unobserved confounding, insufficient support, or critic misspecification can bias or distort the estimate.
- Offline effect coverage: Offline RCE excludes ranking-induced changes in user activity and impression count because proxy values are calibrated to a fixed impression count.It should therefore be interpreted as alignment at the reference impression count, while online A/B testing measures the end-to-end effect.
- Actor parameterization: The actor uses context-dependent convex combinations of selected upstream scores because direct proxy-score prediction was difficult to optimize stably.This improves stability and interpretability but reduces expressiveness and prevents recovery of information absent from upstream scores.
- Evaluation scope: Evaluation is limited to one e-commerce search system and objective horizons of up to four days.Additional platforms, objectives, and longer horizons are needed to establish broader generalizability.
7 Conclusion
DCEO addresses the mismatch between item-level ranking and user-level long-term objectives by learning context-dependent proxy scores with actor-critic optimization. Offline analyses support its effectiveness and interpretability, and a 41-day online A/B test reports a 0.36% GMV improvement over the conventional GMV proxy.
- Framework: DCEO learns context-dependent item-level proxy scores, aggregates them into a user-level metric, and directly optimizes their relative causal effect on the ultimate objective.Only the actor is deployed online, where its learned score is added to the existing multi-objective fusion formula.
- Results: 0.36% GMV improvement over the conventional GMV proxy was reported in a 41-day online A/B test.The paper also reports extensive offline analyses demonstrating effectiveness and interpretability.
8 Ethical Considerations
The study uses behavioral logs and user-side features that may contain sensitive information about interests and purchasing behavior. It reports aggregate results but notes privacy, bias, and exposure risks requiring responsible safeguards.
- Data governance: Behavioral logs and user-side features may contain sensitive information about users’ interests and purchasing behavior.Responsible use requires access controls, data minimization, de-identification, and aggregation.
- Bias and exposure: DCEO may inherit biases from historical interactions, upstream models, and the existing ranking system, while changing item exposure.The passage identifies these as ethical considerations associated with deployment.