PaperlayerView exact revision on arXiv

Source-linked AI summary

UniCon: A Unified Context-Centric Modeling Paradigm for CTR Prediction

Jiajun Cui, Zhengqi Xu, Fan Zhang, Zhangteng, Gu Tang, Honghong Zhu, Mengxi Wu, Yulin Liang, Xingxing Wang

arXiv:2609.03290v2cs.IR

TL;DR

Existing CTR models often unify sequential and non-sequential signals at the token level, despite historical behavior and current requests sharing a context structure. UniCon models history and targets as context units with local and cross-context interactions. In Meituan search advertising, it reports offline AUC and significant online gains over the production baseline.

  • Problem

    Unified CTR models commonly organize sequential and non-sequential signals separately, although historical contexts and the current target share an internal structure and differ chiefly in whether outcomes are observed.

    Source: §1 · pp. 1–2

  • Method

    UniCon organizes history and current candidates as context units, using intra-context interactions for Locality and inter-context interactions for Dynamics.

    Source: §1 · p. 2

  • Results

    Offline AUC improves by 0.0139 over the production base model, and online RPM, CTR, and revenue lift by 3.09%, 2.07%, and 2.95%, respectively.

    Source: §1 · p. 2, §4.9 · p. 8

  • Takeaways & Limitations

    Offline and online gains support context-level unification for context-rich industrial ranking.

    Source: §5 · p. 8

Abstract

from arXiv · show

Unified modeling has become a major direction for industrial click-through rate (CTR) prediction. Existing approaches typically unify sequential and non-sequential signals at the token level, model their interactions in a shared backbone, and increase model capacity to improve scaling behavior. However, this division originates from legacy feature-engineering practice and is misaligned with the underlying decision process. User behavior is inherently a sequence of homogeneous context units; at the level of input organization, historical behavior and the current request differ only in whether their outcomes are observed or remain to be predicted. Treating them as heterogeneous signals obscures structural dependencies within the user's decision context, limiting both scaling efficiency and prediction quality. This limitation is particularly pronounced in context-rich scenarios such as e-commerce shelves and waterfall feeds. To address this, we propose UniCon, a unified context-centric modeling architecture that treats the request context as the basic modeling unit and organizes history and prediction targets as homogeneous context units. Intra-context attention captures local coupling among items within a context (Locality), while inter-context attention models the dynamic evolution of decision states across contexts (Dynamics). This organization bridges the structural gap between history and target and supports more effective scaling of unified CTR models. Context-unit-level sequence compression further reduces deployment overhead. On Meituan search advertising, UniCon improves offline AUC by 0.0139 over a strong production baseline and achieves statistically significant online lifts of 3.09% in RPM, 2.07% in CTR, and 2.95% in revenue.

1 Introduction

UniCon organizes observed history and current candidates as context units, modeling local interactions within contexts and evolving states across them. It combines this organization with context-level compression and reports offline and online gains in Meituan search advertising.

1 Introduction

  • Historical behavior and the current target share an internal context structure; their main difference is whether outcomes are observed or still to be predicted.Separating them into sequential and non-sequential branches can obscure context boundaries and within-display relationships.Source: §1 · pp. 1–2
  • UniCon models each request context as a unit, with intra-context interactions capturing Locality and inter-context interactions capturing Dynamics.The target unit is initialized from current candidates and contextual signals, while its outcomes remain to be predicted.Source: §1 · p. 2
  • Offline AUC improves by 0.0139 over the production base model; online RPM lifts 3.09%, CTR 2.07%, and revenue 2.95%.The online lifts are statistically significant.Source: §1 · p. 2
  • Context-level sequence compression progressively retains historical contexts most relevant to the target, alongside padding-free attention and compiled dynamic-shape deployment.These designs support training and serving under industrial constraints.Source: §1 · p. 2
  • The stated contributions include a shared structural schema for history and target candidates and a hierarchical architecture for within- and cross-context interactions.The paper also identifies scalable implementation and industrial validation as contributions.Source: §1 · p. 2

2 Related Work

Prior unified CTR models commonly share token-level backbones, while context-aware approaches preserve selected forms of decision or display context. UniCon instead makes the context boundary the organizing unit for both inputs and interactions.

2.1 Unified Architectures for CTR Prediction

  • Scalable feature-interaction backbones enlarge ranking models, while unified sequential and non-sequential models share or couple Transformer-style paths; both retain token-centered organization.The former leave behavior sequences outside their primary scope or do not explicitly group items from the same impression; the latter leave impression boundaries implicit [13] [17] [18] [19] [14] [15] [16] [21].Source: §2.1 · p. 2
  • UniCon unifies at the context level, processing observed historical lists and the target latent context through the same stacked architecture.This makes context locality and cross-context dynamics explicit while retaining a shared scalable backbone.Source: §2.1 · p. 2

2.2 Context Modeling for CTR Prediction

  • Prior context-aware work models instance, candidate, session, page, or list context, but often retains behavior sequences, target items, or feature fields as the primary input abstraction.Examples include session-based interest modeling and page- or list-aware modeling.Source: §2.2 · pp. 2–3
  • UniCon treats context as a structural unit that determines signal grouping, intra- and inter-context interactions, variable-length execution, and compression.Figure 1 depicts observed display lists and the candidate-initialized target unit as a dynamic context sequence.Source: §2.2 · p. 3, Figure 1 · p. 3

3.1 Overview

UniCon organizes historical and target inputs as context units and processes them with stacked blocks. The overview combines within- and cross-context interaction, target-aware compression, and multi-task prediction.

3.1 Overview

  • Historical and target inputs are organized as context units in stacked UniConBlocks, with intra-context and inter-context interaction stages.The blocks alternate the stages across the context sequence.Source: §3.1 · p. 3
  • Target-aware context compression selects historical contexts between blocks, while the model uses multi-task prediction.The overview describes compression and prediction as components of the overall architecture.Source: §3.1 · p. 3
  • The architecture combines modeling within each context with modeling across contexts rather than treating the inputs as a flat token sequence.The passage identifies these stages as intra-context and inter-context interaction.Source: §3.1 · p. 3

3.2 Context-Unit Formulation

UniCon represents a request as observed historical context units followed by a candidate-initialized target unit, using a shared schema without treating candidates as observed impressions. Its tokenization, stacked interaction blocks, and auxiliary objectives support this structural alignment.

3.2 Context-Unit Formulation

  • Each instance contains observed historical context units followed by one candidate-initialized target latent unit, preserving historical display structure in a shared hierarchy.The target unit is initialized because the final display list is unavailable before ranking.Source: §3.2 · p. 3, §3.2.2 · p. 4

3.2.1 Context-Unit Representation

  • A context unit combines item-side, contextual, and user-side inputs; the user-side features are aggregated into one user token.Inputs include item identity and order, signals such as query and environment, and primarily dynamic user statistics.Source: §3.2.1 · p. 3
  • Historical item tokens carry logged click feedback, while target candidate tokens use a learnable placeholder in the aligned feedback component.The placeholder is optimized jointly with the model and carries no future click information.Source: §3.2.1 · p. 3
  • Linear tokenizers map item, contextual, and user feature groups into tokens, which the stacked blocks encode within each unit before modeling interactions across units.The block architecture also places target-aware context compression between blocks.Source: §3.2.1 · pp. 3–4, Figure 2 · p. 4

3.2.2 Historical–Target Structural Alignment

  • Exposure and absolute-position supervision refine the candidate-initialized target unit toward logged display-list structure, making its representation more informative for CTR prediction.Historical and target units pass through the same architecture, but the candidate pool is not treated as an observed display list.Source: §3.2.3 · p. 4, §3.2.2 · p. 4

3.2.3 Target-Context Supervision

  • The click, exposure, and absolute-position heads use different training populations: click and position losses apply to exposed candidates, while exposure loss covers all candidates.Exposure predictions are not fed into the click head, and click-loss gradients are blocked from the position head; exposure and position objectives regularize toward the incumbent policy's logged display distribution.Source: §3.2.3 · pp. 4–5
  • At serving time, UniCon evaluates each candidate at every feasible absolute position to produce position-conditioned CTR estimates.The click head is conditioned on an embedding of logged absolute position during training.Source: §3.2.3 · p. 4

3.3 Unified Contextual Modeling Architecture

UniCon organizes observed history and target candidates as context units in one shared sequence, then alternates local within-context modeling with cross-context interaction.

3.3.1 Shared Context Sequence Representation.

  • Historical contexts and the target latent context unit share the stacked architecture, while candidate representations feed task-specific click, exposure, and position heads.Historical units retain logged boundaries and feedback; the candidate pool is not treated as an observed display list.Source: §3.3.1 · p. 5

3.3.2 Hierarchical Intra- and Inter-Context Interaction.

  • Intra-context attention independently processes each context unit, capturing Locality among its items, feedback, user token, and contextual signals.Historical feedback is interpreted relative to the same display's alternatives and conditions.Source: §3.3.2 · p. 5
  • Inter-context attention connects context units in chronological order, allowing history and target to exchange information about changing interests and display conditions.Chronology helps distinguish persistent preferences from context-induced choices.Source: §3.3.2 · p. 5, §3.3 · p. 5

3.4 Efficient Computation and Production Inference

UniCon reduces computation through progressive context compression and variable-length attention, alongside serving optimizations for industrial inference.

3.4.1 Context-Aware Sequence Compression.

  • Relevance-based TopK compression keeps a fixed fraction of historical contexts after the full first block, bounding attention cost by O(N^2/(1-r^2)) rather than O(LN^2).The relevance approximation uses target queries and historical-context keys; serving fuses selection and context gathering to reduce intermediate memory traffic and kernel launches.Source: §3.4.1 · pp. 5–6

3.4.2 Variable-Length Attention Operator.

  • Variable-length attention uses segment boundaries to isolate each context for intra-context attention and the whole instance for inter-context attention, avoiding padding.These boundaries support context-unit isolation for Locality and instance-level interaction for Dynamics.Source: §3.4.2 · p. 6

3.4.3 Production Inference.

  • Production inference combines 300-candidate sharding, graph-compatible feature extraction, parallel dense MMoE kernels, and AOT-compiled dynamic-shape serving.The compiled runtime removes Python from online inference while preserving variable-length execution across requests.Source: §3.4.3 · p. 6

4 Experiments

Experiments compare UniCon with production, unified, and context-enhanced baselines, then test its modeling choices, scaling, compression, serving efficiency, and online impact.

4.1 Dataset and Metrics

  • Meituan evaluation uses one year of chronological production data, with the next two days as validation and test; baselines share UniCon’s data, raw fields, and training resources.Models were trained on 16 NVIDIA A100 GPUs with batch size 1600; the main offline and online experiments use compressed UniCon-Large with rℓ=0.5.Source: §4.1 · p. 6, §4.2 · p. 7, §4.3 · p. 7

4.4 Overall Performance

  • 197.14 GFLOPs: compressed UniCon-Large retains the strongest overall offline quality, and UniCon beats scaling and context-enhanced baselines across AUC, LogLoss, and GAUC.Table 1 compares prediction quality, parameter count, and inference computation; AUC measures ranking quality, LogLoss calibration, and GAUC request-level ranking quality.Source: Table 1 · p. 7, §4.4 · p. 7, §4.1 · p. 7

4.5 Ablation Studies

  • Flattening historical and target tokens causes the largest ablation degradation; removing explicit context organization, a shared target interface, or intra-context interaction also worsens metrics.The flattened-backbone and intra-context comparisons match parameter count, and the latter also preserves Transformer depth.Source: §4.5 · pp. 7–8, Table 2 · p. 8
  • Exposure prediction contributes more than absolute-position prediction, and removing both causes the largest auxiliary-task drop; compression matches uncompressed performance within run variation at far lower computation.Compression also outperforms time-based truncation, favoring target-aware selection over a fixed recency rule.Source: §4.5 · pp. 7–8

4.6 Scaling Experiments

  • AUC rises from 0.8668 to 0.8698 as UniCon grows from approximately 0.05B to 0.33B parameters, while retaining an advantage over representative baselines across comparable sizes.The scaling comparison plots AUC against parameter count on a logarithmic scale.Source: §4.6 · p. 8, Figure 3 · p. 8

4.7 Compression Trade-off

  • 75.4% less profiled computation: reducing the keep ratio from 1.0 to 0.5 changes AUC from 0.8698 to 0.8697; more aggressive compression lowers AUC further.AUC is 0.8695 at r=0.4 and 0.8692 at r=0.3; the final blocks then retain nearly one historical context unit, weakening cross-context interaction.Source: §4.7 · p. 8, Figure 4 · p. 8

4.8 Serving Efficiency

  • 258.1% higher throughput than the fixed-length, uncompressed baseline: variable-length attention alone gains 64.5%, and adding compression raises the gain by 117.6%.The combined serving configuration differs in AUC by 0.0001 and meets the latency SLA despite higher GFLOPs than the production Base.Source: §4.8 · p. 8

4.9 Online A/B Testing

  • 3.09% RPM, 2.07% CTR, and 2.95% revenue lifts were significant (p≤0.01) in a seven-day Meituan search-advertising A/B test.The test assigned 20% of traffic to UniCon-Large; latency remained within the production SLA, with no significant degradation in other guardrail metrics.Source: §4.9 · p. 8

5 Conclusion

UniCon organizes observed history and current candidates under a shared context-unit schema, models Locality and Dynamics, and combines auxiliary supervision with scalable implementation to support gains in context-rich industrial ranking.

5 Conclusion

  • UniCon places observed history and current candidates under a shared context-unit schema, with intra- and inter-context layers modeling Locality and Dynamics.Source: §5 · p. 8
  • Auxiliary supervision refines the target representation.Source: §5 · p. 8
  • UniCon outperforms unified and context-enhanced baselines across parameter scales while staying within the production SLA.Offline and online gains support context-level unification for context-rich industrial ranking.Source: §5 · p. 8

A Intra- and Inter-Context Block Implementation

The intra- and inter-context stages share a pre-normalized residual-block structure but differ in their attention boundaries and processing of context units.

A Intra- and Inter-Context Block Implementation

B Impact of Candidate Sharding

Candidate sharding bounds computation and memory at the cost of restricting interactions to each shard; smaller shards can weaken contextual evidence for candidates.

B Impact of Candidate Sharding

Loading 2609.03290v2…