Source-linked AI summary
Towards a Unified Framework of Optimal Input-Side Dimensional Configuration for Deep CTR Prediction
Chaohua Yang, Fuyuan Lyu, Xing Tang, Dugang Liu, Xiuqiang He, Zhong Ming
TL;DR
Input-side sample, feature, and embedding-unit selection is usually optimized independently, despite shared forms and dependencies across levels. OptIDC jointly learns these configurations and coordinates them with selection-weighted optimal transport. Experiments report improved performance over existing input-side selection methods across four real-world datasets.
Problem
Existing input-side selection methods primarily optimize sample, feature, or embedding-unit selection independently, overlooking their shared form and dependencies across levels.
Source: §1 · pp. 1–2Method
OptIDC jointly learns sample, feature, and embedding-unit configurations and coordinates cross-level selections using selection-weighted optimal transport over finite measures.
Source: §1 · p. 2Results
Across evaluated settings, joint configuration improves over corresponding single-level alternatives, and OT coordination provides further gains over joint configuration alone.
Source: §1 · p. 2Takeaways & Limitations
OptIDC configures input information at three levels while leaving the backbone interaction architecture unchanged, allowing use with different CTR models.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Deep click-through rate (CTR) prediction has achieved substantial success through increasingly expressive feature interaction architectures. However, predictive performance also depends on how input-side information is configured before interaction modeling. Existing studies have shown that selecting informative samples, features, or embedding units can improve prediction accuracy. Nevertheless, they primarily optimize these selection mechanisms independently along a single input-side dimension, overlooking their shared basic form and the inherent dependencies across input levels, thereby limiting the effectiveness of input-side dimensional configuration. To address this issue, we propose OptIDC, a unified framework for Optimal Input-side Dimensional Configuration in deep CTR prediction. OptIDC jointly learns sample, feature, and embedding-unit level configurations within a unified framework by integrating their corresponding selection mechanisms. To explicitly coordinate selection decisions across different levels, OptIDC further introduces a selection-weighted Optimal Transport mechanism that aligns coarse- and fine-grained selections via transport over finite measures, thereby alleviating cross-level selection conflicts while preserving the semantics of each input level. Moreover, OptIDC is model-agnostic and can be seamlessly integrated into existing CTR models. Extensive experiments on four real-world benchmark datasets demonstrate that OptIDC consistently achieves higher AUC than existing input-side selection methods, validating the effectiveness of unified input-side dimensional configuration.
1 Introduction
CTR prediction depends both on how architectures model interactions and on which input information shapes the model. OptIDC jointly configures samples, features, and embedding units, coordinating their selections across levels; experiments report gains over single-level alternatives and from OT coordination.
1 Introduction
- Prior selection methods usually optimize one input level at a time, leaving sample, feature, and embedding-unit decisions without explicit coordination.These decisions ultimately shape the same model input, but differ in granularity and structure.Source: §1 · pp. 1–2
- Sample, feature, and embedding-unit selection share a masking form but are dependent: suppressing a feature can make its unit-level decisions ineffective.A highly weighted sample can likewise contribute little if most of its feature representations are suppressed.Source: §1 · p. 2
- OptIDC jointly learns sample, feature, and embedding-unit configurations and coordinates their pairwise relations using selection-weighted optimal transport over finite measures.Coarse-level weights set transported mass, finer-level weights allocate it, and representation dissimilarity defines transport cost.Source: §1 · p. 2
- Across evaluated settings, joint configuration improves over corresponding single-level alternatives, and OT coordination adds gains over joint configuration alone.The framework was evaluated on four public datasets and five representative backbones.Source: §1 · p. 2
2 Related Work
Prior CTR work advances interaction modeling and automates selection across several input and modeling dimensions. OptIDC differs by jointly configuring sample, feature, and embedding-unit inputs.
2 Related Work
- Deep CTR architectures model complex feature interactions, while typically treating samples, features, and embedding units equally during modeling.The passage describes factorization machines, Wide&Deep, DeepFM, interaction networks, attention, and multi-task learning as examples of architectural advances, [2] [7], [27].Source: §2 · p. 2
- Unlike methods that usually optimize one selection space at a time, OptIDC jointly considers sample, feature, and embedding-unit levels.It complements architectural advances by configuring information at these three input-side levels.Source: §2 · pp. 2–3
3 Preliminaries
Motivating experiments find larger AUC gains when the three input levels are configured together, and an association between selection consistency and performance. The formulation treats sample, feature, and embedding-unit selectors as coupled decisions and introduces explicit cross-level coordination.
3.1 Motivating Experiments
- About 0.30 higher SCR across the examined cases accompanies nearly 0.008 AUC gain, an association that motivates explicit coordination but is not causal evidence.SCR is bounded within [0,1], with larger values indicating greater agreement in support and mapped activation magnitude.Source: §3.1 · p. 3, Figure 2 · p. 3
- More than 0.002 AUC gain for IDC, versus about 0.001 for each single-level variant, on Avazu with a DNN backbone.IDC learns all three selectors under the CTR prediction objective without explicit cross-level coordination.Source: Figure 2 · p. 3, §3.1 · p. 3
3.2 Problem Formulation
- Sample weights, feature gates, and embedding-unit masks respectively control each instance's learning contribution, selected fields, and selected coordinates.The feature and unit decisions configure the embedding input; the sample weight then weights its loss.Source: §3.2 · p. 3
- IDC shares the prediction loss across selectors but lacks a direct term relating coarse decisions to the finer decisions through which they take effect.The formulation considers sample–feature, feature–embedding-unit, and sample–embedding-unit relations, with costs measuring whether coarse decisions are supported by compatible fine-grained components.Source: §3.2 · p. 3
4 The OptIDC Framework
OptIDC jointly configures sample, feature, and embedding-unit inputs, and coordinates their decisions through selection-weighted optimal transport. It searches configurations with continuous gates, then retrains the backbone on fixed hard masks.
4.1 Framework Overview
- OptIDC learns sample, feature, and embedding-unit configurations together, linking each coarse-to-fine selector pair through selection-weighted optimal transport.Its coordination cost uses finite measures and representation compatibility while preserving each selector’s native granularity.Source: §4.1 · p. 4, §4.3 · pp. 4–5
4.2 Unified Input Configuration Learning
- Feature and embedding-unit gates configure the embedding tensor before a sample selector scores it to determine how strongly the example contributes to the loss.The feature gate controls a whole embedding, while the unit gate controls individual coordinates within it.Source: §4.2 · p. 4
4.3 Cross-Level Coordination via OT
- For each coarse-to-fine relation, the coarse score sets transported mass, normalized fine-level weights allocate it, and representation dissimilarity supplies the transport cost.The three relations are sample–feature, feature–embedding-unit, and sample–embedding-unit; their costs penalize selected coarse components whose fine allocations have high representation cost.Source: §4.3 · pp. 4–5
4.4 Optimization
- Search uses sparsity on feature and unit gates plus a sample-retention ratio constraint; retraining removes auxiliary selectors and fits the backbone with hard masks fixed.The sample selector is used only to construct the retraining set and adds no serving-time overhead.Source: §4.4 · pp. 5–6
5 Experiments
The experiments evaluate OptIDC across four public datasets and five CTR backbones, addressing performance, ablation, efficiency, design choices, and selection consistency. Results compare OptIDC with single-level methods and uncoordinated joint selection, then examine its costs and coordination behavior.
5.1 Experiment Setup
- The evaluation uses Avazu, Criteo, MovieLens-1M, and AliCCP, and combines methods with five backbones: DNN, DeepFM, DCN, AutoInt, and FinalMLP.AUC and LogLoss are the evaluation metrics.Source: §5.1 · p. 6
5.2 RQ1: Performance Comparison
- OptIDC achieves significant AUC improvements over IDC on all datasets, with a clearer margin on Avazu.The comparison covers Avazu and Criteo in Table 1 and the other two datasets in Table 5.Source: §5.2 · p. 6, Table 1 · p. 7
5.3 RQ2: Ablation Study
- Adding the corresponding OT relation improves AUC and improves or maintains LogLoss for each two-level model across five backbones; removing any OT term lowers AUC and increases LogLoss on Avazu.OptIDC also improves both metrics over IDC and outperforms AOP.Source: §5.3 · p. 7, Table 3 · p. 8
5.4 RQ3: Efficiency Analysis
- On Avazu, OptIDC requires 2575 seconds total training time and 0.1032 seconds per-batch training time, with 0.0041 seconds per-batch inference time.Per-batch training is 4.3%–14.4% higher than OptEmbed, OptFS, and AutoDenoise, but lower than AutoDim; inference is comparable to those three methods. Figure 4 compares model size across four datasets.Source: §5.4 · p. 7, Table 2 · p. 7, Figure 4 · p. 8
5.5 RQ4: In-depth Analysis of OptIDC
- On Avazu, OptIDC’s cosine-based ground cost performs better than weight- and embedding-distance costs, while cosine- and KL-based direct coordination methods yield only marginal gains over IDC.OptIDC performs better across all five backbones; the compared coordination relations have supports fixed by the three configuration levels.Source: §5.5 · p. 8, Figure 5 · p. 8
5.6 RQ5: Selection Consistency Analysis
- With OT coordination, OptIDC has higher SCR than IDC for every backbone on both Avazu and Criteo, indicating greater agreement among the learned configurations.SCR compares masks after mapping them into a common space because the selection levels operate at different granularities.Source: §5.6 · p. 8, Figure 6 · p. 9
6 Conclusions
OptIDC jointly configures samples, features, and embedding units, coordinating their selections with optimal transport. The appendix describes mappings that represent selection masks as embedding activations.
6 Conclusions
- OptIDC jointly models sample, feature, and embedding-unit selection and uses an Optimal Transport-based strategy to coordinate configuration across levels.Experiments on four real-world datasets are reported to show improved performance through more effective identification of critical information.Source: §6 · p. 9
A Mask Mapping Mechanism
- The sample mask defines the retained sample set used to calculate feature frequencies for selected samples.These frequencies support the sample-mask mapping to embedding activation.Source: A Mask Mapping Mechanism · p. 10
- Sample selection is remapped through retained-sample feature frequencies, which serve as embedding-unit activation frequencies because embeddings update when corresponding features appear.The remapping expresses the sample mask as an embedding-unit activation matrix for comparison with feature- and embedding-unit selection.Source: A Mask Mapping Mechanism · p. 10
- Feature-mask mapping remaps G to an embedding-unit activation matrix using log-scaled original feature frequencies broadcast across embedding dimensions.The log operation reduces the influence of high-frequency features.Source: A Mask Mapping Mechanism · p. 10
- Embedding-unit selection is remapped to an activation matrix using the activation frequency of each embedding unit.Source: A Mask Mapping Mechanism · p. 10
B.1 Dataset Preprocessing and Statistics
The preprocessing applies dataset-specific feature filtering and transformations, then divides the datasets into training, validation, and test sets with dataset-specific procedures.
B.1 Dataset Preprocessing and Statistics
- For Avazu and Criteo, features below frequency 2 are filtered; Avazu also drops instance ID and transforms timestamp into hour, weekday, and is_weekend fields.Both datasets are randomly split 8:1:1.Source: §B.1 · p. 10
- MovieLens-1M ratings are converted to clicks at threshold 4, without low-frequency filtering, and randomly split 8:1:1.Ratings of 4 or higher indicate a click.Source: §B.1 · p. 10
- For AliCCP, features below frequency 10 are filtered; 10% of the original training set is sampled for validation, and the original test set is retained.Source: §B.1 · p. 10
B.2 Implementation Details
OptIDC improves AUC over IDC across the evaluated datasets, while the implementation uses shared training settings and five-seed reporting for selected results.
B.2 Implementation Details
- The implementation fixes embedding size at 16 and batch size at 4096, uses three-layer MLP components, and tunes learning-rate and regularization settings.The compared experiments use five random seeds; Tables 1 and 5 report mean ± standard deviation, while other results report means.Source: §B.2 · pp. 10–11
C.1 Performance Comparison
- OptIDC achieves significant AUC improvements over IDC on all datasets, with a clearer margin on Avazu.On AliCCP, OptIDC has the highest AUC across all five backbones but not the lowest LogLoss.Source: §C.1 · p. 11
- Single-level methods often outperform the Backbone, and IDC is generally stronger than single-level alternatives, though benefits vary by backbone and dataset.The reported variation is consistent with different redundancy in samples, feature values, and embedding units.Source: §C.1 · p. 11
C.2 Ablation Study
Criteo ablations support three-level configuration and OT coordination; the coordination alternatives vary the transport cost or compare broadcast selection vectors directly.
C.2 Ablation Study
- For three-level configuration, OptIDC improves both AUC and LogLoss over IDC across all five backbones; removing any OT term lowers AUC across all five.LogLoss increases or remains unchanged when an individual OT term is removed.Source: §C.2 · p. 11
C.3 Details of Coordination Alternatives
- The coordination alternatives sum three relation penalties and average over samples, with the OT penalty independently tuned while other objective terms remain unchanged.Source: §C.3 · p. 11
- The wWED and wEED variants replace only ground costs, retaining selection masses and allocations; wWED compares scalar weights, while wEED compares normalized representations.The compared representation pairs correspond to sample–feature, feature–unit, and sample–unit relations.Source: §C.3 · p. 11
- The wCS and wKL alternatives replace relation weights with direct comparisons of broadcast selection vectors.Source: §C.3 · p. 11
C.4 Selection Ratio Analysis
On Avazu, OptIDC achieves superior predictive performance for both sample selection and embedding-parameter selection while generally retaining fewer samples and embedding parameters than most baselines.
C.4 Selection Ratio Analysis
- OptIDC consistently achieves superior predictive performance for both sample selection and embedding-parameter selection.The comparison uses three backbone models on Avazu.Source: §C.4 · p. 11
- OptIDC generally retains fewer samples and embedding parameters than most baselines.Embedding-parameter retention is jointly determined by feature selection and embedding-unit selection.Source: §C.4 · p. 11
- The analysis compares sample and embedding retention ratios between OptIDC and baselines on Avazu.Embedding-parameter retention reflects feature selection together with embedding-unit selection.Source: §C.4 · p. 11
C.5 Hyperparameter Sensitivity Analysis
On Avazu, OptIDC’s sensitivity trends are broadly similar across three backbones; performance is relatively stable over parts of the tested ranges, but large feature- and embedding-unit sparsity penalties can substantially lower AUC.
C.5 Hyperparameter Sensitivity Analysis
- The analysis varies five key hyperparameters across their predefined ranges using three backbone models on Avazu.The parameters are λco, λrat, λent, λf, and λe.Source: §C.5 · p. 11
- On Avazu, AUC remains relatively stable over parts of the tested ranges, but large feature- and embedding-unit sparsity penalties can substantially degrade it.The sensitivity trends are broadly similar across the three backbone models.Source: §C.5 · p. 12, Figure 8 · p. 13
- Figure 8 plots AUC against the search range of each hyperparameter.The parameters are varied within their respective predefined ranges.Source: §C.5 · p. 12, Figure 8 · p. 13