PaperlayerView exact revision on arXiv

Source-linked AI summary

LIFT: A Lifecycle-aware Interaction Factorization Transformer for Unified Retrieval and Ranking

Keji Miao, Enhao Cheng, Yan Li, Qun Li, Qi Zhang, Jie Yuan, Xiaoyong Li

arXiv:2610.10556v1cs.IR

TL;DR

Retrieval and ranking share user history but have different information available within an interaction, a distinction left implicit by item- or event-level sequential states. LIFT factorizes each interaction into ordered Request, Item, Context, and Action states in a causal sequence, with retrieval reading Request and ranking reading Context. LIFT’s Joint Score reaches 0.2724 on ML-20M, improving over HSTU by 4.9%, and 0.2239 on Taobao, improving over UniPinRec by 3.6%.

  • Problem

    Retrieval and ranking share user history but have different information available within an interaction, a distinction left implicit by item- or event-level sequential states.

    Source: §1 · pp. 1–2

  • Method

    LIFT factorizes each interaction into ordered Request, Item, Context, and Action states in a causal sequence, with retrieval reading Request and ranking reading Context.

    Source: §1 · p. 2

  • Results

    LIFT’s Joint Score reaches 0.2724 on ML-20M, improving over HSTU by 4.9%, and 0.2239 on Taobao, improving over UniPinRec by 3.6%.

    Source: §4.1 · p. 8

  • Takeaways & Limitations

    Loss-weight sweeps show favorable retrieval–ranking trade-offs for LIFT, while lifecycle sequence ablations and scaling analyses examine design and capacity effects.

    Source: §5 · p. 10

    The four-role decomposition follows field-generation dependencies and information availability, independently of how fields are organized in a data table.

    Source: §3.2 · p. 5

Abstract

from arXiv · show

Cascaded recommender systems use the same user history for retrieval and ranking, while the two stages have access to different information at different points of an interaction. We propose the Lifecycle-aware Interaction Factorization Transformer (LIFT), which decomposes each interaction into ordered Request, Item, Context, and Action states and models them as a causal sequence. Retrieval reads the Request state, while ranking reads the Context state, allowing both tasks to share history modeling while preserving stage-specific information. LIFT instantiates this representation with Role-Conditioned Attention and a lightweight Pre-LN Bias. On ML-20M and Taobao, LIFT achieves the highest Joint Score among the evaluated joint models, improving over the strongest baselines by 4.9% and 3.6%, respectively. Loss-weight sweeps show favorable retrieval--ranking trade-offs, while ablations and scaling analyses further examine lifecycle sequence construction, model components, and capacity settings.

1 INTRODUCTION

Retrieval and ranking reuse user history but operate with different information available at different points in an interaction. LIFT represents those intermediate states explicitly so the stages can share causal history modeling while reading stage-aligned states.

1 INTRODUCTION

  • The interaction-level states address implicit information-availability boundaries in item- or event-level sequential representations, despite retrieval and ranking sharing user history.Retrieval uses the history and current request; ranking also has the candidate and contextual features.Source: §1 · pp. 1–2
  • Each interaction becomes an ordered Request, Item, Context, and Action sequence; retrieval reads Request, while ranking reads Context.The states follow when information becomes available during an interaction.Source: §1 · p. 2
  • Online serving prefills the historical sequence once and reuses its cached states for retrieval and ranking.Retrieval and ranking read the Request and Context states, respectively.Source: §1 · p. 2
  • Role-Conditioned Attention and a lightweight Pre-LN Bias instantiate the shared causal history encoder while retrieval and ranking retain task-specific processing and prediction heads.Attention shares Query and Key projections while conditioning Value and Output transformations on lifecycle roles; Pre-LN Bias conditions the feed-forward path on the current role.Source: §1 · p. 2
  • LIFT achieves the highest Joint Score among evaluated joint models, improving over the strongest baselines by 4.9% on ML-20M and 3.6% on Taobao.The comparisons cover single-task, multi-task, and unified sequence models.Source: §1 · p. 2

2 RELATED WORK

Related work spans sequential behavior modeling, unified retrieval–ranking architectures, and multi-task parameter sharing. LIFT distinguishes its approach by exposing multiple causal states within each interaction and aligning task readouts to lifecycle stages.

2.1 SEQUENTIAL RECOMMENDATION MODELING

  • Sequential recommendation moves from recurrent encoders such as GRU4Rec to self-attention in SASRec and later Transformer extensions; LIFT makes the request a distinct token.The request token explicitly uses request-side information available at retrieval time.Source: §2.1 · pp. 2–3

2.2 UNIFIED MODELING OF RETRIEVAL AND RANKING

  • Unified retrieval–ranking work includes shared generative frameworks and cross-stage caching; LIFT instead exposes Request, Item, Context, and Action as distinct causal states.The states follow information availability, allowing retrieval and ranking to read different points in the same causal history.Source: §2.2 · p. 3

2.3 MULTI-TASK LEARNING AND STRUCTURED PARAMETER SHARING

  • Multi-task methods share or specialize parameters through shared-bottom models, MMoE, and PLE; LIFT shares a causal interaction sequence but gives retrieval and ranking different state readouts.The readouts are suited to the respective recommendation stages.Source: §2.3 · p. 3

3 METHOD

LIFT shares a causal history sequence across retrieval and ranking while giving each stage a readout aligned with the information available at that point. It represents interactions as ordered lifecycle states and uses role-conditioned transformations to model them.

3.1 PROBLEM SETUP AND SYSTEM OVERVIEW

  • Retrieval reads Request states and ranking reads Context states from shared causal history encoding, while candidate processing and prediction interfaces remain task-specific.At serving time, the history is prefilled once and cached states are reused by both stages.Source: §3.1 · pp. 3–4

3.2 LIFECYCLE-AWARE INTERACTION REPRESENTATION

  • Each interaction is decomposed into Request, Item, Context, and Action fields and serialized in lifecycle order, with role-specific inputs projected into a shared hidden space.The decomposition follows field-generation dependencies and information availability rather than the organization of a data table.Source: §3.2 · p. 5
  • Causal ordering makes the Request state depend on preceding information, while the Context state can additionally use the current interaction’s Item and Context; the target Action is excluded from input.These distinct information sets form the retrieval and ranking readouts, respectively, without collapsing their stage boundary.Source: §3.2 · p. 6

3.3 ROLE-CONDITIONED ATTENTION AND PRE-LN BIAS

  • Role-Conditioned Attention shares Query and Key projections while conditioning Value and Output transformations on the source and receiving roles, respectively.Queries and Keys use normalized states augmented with role information; Values use normalized states with role-specific projections.Source: §3.3 · pp. 6–7
  • LIFT adds a role-specific bias on the pre-normalization path to the feed-forward sublayer.The final Request and Context states are passed to retrieval and ranking heads, respectively.Source: §3.3 · p. 7

4 EXPERIMENTS

Across ML-20M and Taobao, LIFT achieves the highest Joint Score among evaluated joint models. Trade-off, ablation, and scaling analyses examine its operating points, components, sequence construction, and capacity.

4.1 EXPERIMENTAL SETUP

  • LIFT’s Joint Score reaches 0.2724 on ML-20M, improving over HSTU by 4.9%, and 0.2239 on Taobao, improving over UniPinRec by 3.6%.Source: §4.1 · p. 8

4.2 RETRIEVAL–RANKING PARETO FRONTIER

  • Across evaluated loss weights, LIFT forms the global empirical Pareto frontier in Recall@100–AUC and NDCG@100–GAUC spaces.Increasing λret shifts LIFT toward retrieval-oriented operating points, while moderate weights preserve stronger ranking quality.Source: §4.2 · pp. 8–9, Figure 3 · p. 9

4.3 ABLATION STUDIES

  • Removing Pre-LN Bias lowers Joint Score from 0.2724 to 0.2672, removing Role-Conditioned Attention to 0.2695, and removing both to 0.2663.The ablation holds the lifecycle sequence, task readouts, heads, and training protocol fixed; the full model performs best on all reported metrics.Source: §4.3 · p. 9, Table 3 · p. 9
  • The fully factorized [R][I][C][A] sequence scores 0.2724, versus 0.2582 for fully merged [RICA] and 0.2639 for the alternative four-token layout.The reported gap between the two four-token layouts suggests that aligning token boundaries with information-availability stages matters for constructing effective task states.Source: §4.3 · p. 9, Table 4 · p. 9

4.4 CAPACITY AND SEQUENCE SCALING ANALYSIS

  • LIFT exceeds UniPinRec at every evaluated depth and hidden dimension; with d = 64, it scores 0.2648 using 13.88M parameters, versus 0.2582 for UniPinRec at d = 256 with 18.23M parameters.At matched four-token sequence length, encoder depth, and hidden dimension, LIFT trains in 979.0 seconds versus 1440.8 seconds for UniPinRec, a 32.1% reduction, and uses 5.50 GB versus 7.16 GB peak memory.Source: §4.4 · pp. 9–10, Figure 4 · p. 10
  • Adding blank tokens leaves UniPinRec’s Joint Score nearly unchanged at 0.2572–0.2596, while LIFT reaches 0.2724 with four semantic lifecycle tokens.The matched-length training-time and peak-memory comparison is reported in the preceding scaling point.Source: §4.4 · p. 10, Figure 4 · p. 10

5 CONCLUSION

LIFT uses lifecycle-aware interaction factorization to share causal history modeling while aligning retrieval and ranking representations with their respective stages. Experiments show improved joint retrieval–ranking quality and favorable task trade-offs.

5 CONCLUSION

  • LIFT shows consistent improvements in joint retrieval–ranking quality on ML-20M and Taobao.Loss-weight sweeps also show favorable trade-offs between retrieval and ranking.Source: §5 · p. 10
  • Each interaction is decomposed into ordered Request, Item, Context, and Action states so retrieval and ranking read stage-aligned representations.Role-conditioned attention and a lightweight Pre-LN Bias instantiate the factorization in a shared Transformer backbone.Source: §5 · p. 10
  • Lifecycle sequence ablations and scaling analyses examine sequence construction and model capacity.The conclusion also notes that Role-conditioned attention and Pre-LN Bias are the model components used to instantiate the factorization.Source: §5 · p. 10

A DATASET FEATURE ASSIGNMENT

The experiments assign lifecycle features using historical information available before the current interaction, while reserving the target Action for supervision. Dataset-specific positive-label rules and historical features are specified.

A DATASET FEATURE ASSIGNMENT

  • Derived historical statistics use only information available before the current interaction.The target Action is excluded from the input at the prediction position and used only to construct supervision.Source: A DATASET FEATURE ASSIGNMENT · p. 14
  • For ML-20M, ratings greater than 4 are positive; for Taobao, purchases are positive and other behavior types are negative.The feature-assignment table covers ML-20M and Taobao.Source: A DATASET FEATURE ASSIGNMENT · p. 14, Table 5 · p. 14
  • Item counts, user activity, user–item counts, average ratings, and time gaps are constructed from historical observations only.Sequence-based models do not expose the current target Action to the encoder.Source: A DATASET FEATURE ASSIGNMENT · p. 14

B HYPERPARAMETER SEARCH SPACE

The experiments use coordinate search to select hyperparameter configurations according to validation performance. The search spaces are summarized in Table 6.

B HYPERPARAMETER SEARCH SPACE

C DEDICATED SINGLE-TASK RESULTS

Under dedicated retrieval and ranking training, LIFT leads retrieval results and ranking AUC on both datasets, with a Taobao GAUC exception. The reported comparison is summarized in Table 7.

C DEDICATED SINGLE-TASK RESULTS

D CAUSAL MASK WITH MULTIPLE EXPOSURES

The multiple-exposure causal mask combines event-level temporal causality, exposure-level isolation, and lifecycle ordering. It prevents co-occurring candidate exposures from being treated as temporally ordered.

D CAUSAL MASK WITH MULTIPLE EXPOSURES

Related papers

Loading 2610.10556v1…