Source-linked AI summary
SWIM: Step-Wise Integrated Measure for Session-supervised List Evaluation in Generative Re-ranking
Yuanhao Pu, Chenghao Zhang, Chao Feng, Xunyong Yang, Xiang Li, Yongqi Liu, Defu Lian, Kaiqiao Zhan, Kun Gai
TL;DR
Conventional re-ranking evaluators score request-level lists independently, overlooking session-level dependencies and user continuation. SWIM models a candidate list’s prefix-conditioned contribution through finite-horizon survival and reached-position rewards, estimated in parallel with a causally masked Transformer. It consistently outperforms strong baselines in offline and online evaluations, while future work is needed for counterfactual evaluation and longer, heterogeneous browsing trajectories.
Problem
Existing re-ranking evaluators aggregate request-level utilities and overlook cross-request dependencies and long-term user continuation in continuous feeds.
Method
SWIM factorizes prefix-conditioned session value into recursive survival probabilities and reached-position conditional rewards, estimating them in parallel with a causally masked Transformer.
Results
SWIM achieves the best performance on both offline datasets and both reported metrics, consistently outperforming pointwise, listwise, G-only, and G-E baselines.
Takeaways & Limitations
Explicitly modeling user continuation and reached-position conditional reward provides a stronger evaluator for request-level re-ranking.
Takeaways & Limitations
Future work must develop robust counterfactual and off-policy evaluation protocols and extend modeling to longer, more heterogeneous browsing trajectories.
Abstract
from arXiv · showhide
Modern industrial recommender systems have increasingly adopted the Generator-Evaluator (G-E) framework for the re-ranking stage. Within this paradigm, the generator produces candidate item lists from a pool filtered by upstream retrieval and ranking modules, while the evaluator scores these lists and selects the highest-scoring one for final exposure per request. However, on sequential platforms (e.g., short-video apps), users consume items continuously, ignoring artificial list boundaries. Conventional evaluators score lists by aggregating point-wise values, implicitly assuming exposure independence. This fails to capture critical session-level dynamics, such as contextual dependencies, user continuation, and diminishing marginal utility from repetitive content. To bridge this gap, we propose SWIM (Step-Wise Integrated Measure), a list-level evaluator that models user behaviors as a finite-horizon prefix session-level survival process. SWIM estimates the prefix-conditioned contribution of the current list to the session-level objective by factorizing it into a recursive survival distribution and reached-position conditional rewards. Leveraging a causally-masked Transformer, SWIM efficiently estimates continuation probabilities and utilities in parallel, satisfying strict industrial latency constraints. Extensive experiments demonstrate that SWIM significantly outperforms baselines in listwise reranking tasks, yielding substantial improvements in overall recommendation engagement.
1 Introduction
Industrial re-ranking evaluators often score request-level lists independently, overlooking continuous session dynamics and user continuation. SWIM addresses this gap with a survival-based, session-aware evaluator designed for efficient deployment and validated by improved engagement outcomes.
- Motivation: Sequential feeds present lists as a continuous browsing experience, while conventional evaluators aggregate point-wise utilities independently for each request.Users perceive an uninterrupted feed despite request-level list boundaries.
- SWIM: SWIM models recommendation behavior as a finite-horizon survival process over interaction prefixes.The formulation captures sequential continuity and decomposes list value into prefix-level contributions.
- SWIM: SWIM factorizes expected list value into recursive survival probabilities and reached-position conditional rewards.The two components represent reaching later interaction steps and the rewards generated at those positions.
- Implementation: A causally masked Transformer estimates continuation probabilities and utilities in parallel for each candidate list.This avoids explicit autoregressive rollouts across positions and supports industrial latency requirements.
- Results: SWIM consistently outperforms strong baselines on long-horizon reward metrics including total dwell time and overall session engagement.The reported validation uses large-scale industrial datasets.
2 Related Works
Related work spans listwise re-ranking, sequential recommendation, and survival modeling. SWIM differs by evaluating how a given list contributes to cumulative session value rather than predicting an individual next item or only modeling intra-request exits.
- Listwise Re-ranking and G-E: Listwise re-ranking models cross-item dependencies, while G-E systems use generators to propose lists and evaluators to select among them.The evaluator remains a bottleneck when candidate lists are scored independently across requests.
- Listwise Re-ranking and G-E: Existing evaluators typically aggregate request-level utilities and therefore overlook cross-request dependencies and long-term user continuation.This independence assumption motivates session-aware evaluation.
- Comparison with CAVE: CAVE models intra-request consumption value, whereas SWIM targets inter-request session continuation induced by the current list.CAVE weights sub-list values with exit probabilities; SWIM uses recursive step-wise survival.
- Sequential Recommendation: SBR and sequential recommendation models infer intent or predict future-item interactions from interaction histories.Examples include recurrent, attention-based, graph-based, and self-attentive architectures.
- Sequential Recommendation: SWIM evaluates the long-term cumulative contribution of a deterministically given candidate list within a continuous session.This differs from asking which individual item should be recommended next.
- Survival Modeling: Prior survival approaches model return times, scrolling budgets, or item-level continuation, whereas SWIM applies survival modeling to session-level list evaluation.The related approaches share an emphasis on continuation and early exits but operate at narrower levels.
3 Preliminaries
The preliminaries formulate recommendation as a continuous session with a stopping position and cumulative reward, then connect that objective to request-level list serving. They define conditional and session-wise current-list values through user reach probabilities and position rewards.
- Session Formulation: A recommendation session is modeled as a continuous stream ending at a discrete stopping position τ, with the user consuming exactly the preceding prefix.The stopping position is the first position not reached or consumed.
- Session Formulation: Total session value is the cumulative reward collected before termination, and the objective is its expected value.Rewards may combine engagement signals such as clicks, likes, shares, follows, and play time.
- Request-level Serving: Request-level serving selects a fixed-length ordered list from an upstream candidate set because latency prevents optimizing an entire session trajectory at once.The current list is therefore a local segment of the global session trajectory.
- Request-level Serving: The current-list re-ranking problem includes estimating how the list affects user survival and session-level value through the observable session prefix.The prefix contains prior interactions, feedback, cumulative engagement, request index, and context.
- Survival Process and Reward: The survival process defines position reach indicators, monotonic continuation, and step-wise probabilities conditioned on the reached list prefix.Combining request-entry and in-list survival yields the session-wise probability of reaching each position.
- Survival Process and Reward: Conditional current-list value weights each position reward by the probability that the user reaches that position after entering the request.Session-wise contribution additionally accounts for the probability of reaching the current request from the preceding prefix.
4 Methodology
SWIM encodes the session prefix and candidate list with a causal Transformer, predicts step-wise survival and reached-position rewards, and combines them into a current-list score for online selection. Its training objective supervises these survival and conditional-reward components.
- Prefix Encoder: SWIM encodes observable prefix features and each candidate list into prefix-aware hidden states for evaluation.The prefix representation includes user history, request index, previous feedback, cumulative engagement, and contextual features.
- Prefix Encoder: A dedicated prefix token represents entry into the current request, while item tokens represent list positions and their prefixes.The prefix token estimates boundary survival; item tokens estimate in-list survival and rewards.
- Survival Estimation: Causal masking ensures each position is encoded only from the session prefix and preceding list positions.The lower-triangular attention mask supports prefix-causal representations.
- Survival Estimation: SWIM predicts step-wise survival probabilities with shared sigmoid heads and recovers session-wise reach probabilities through cumulative products.The boundary probability is predicted from the prefix state, while later probabilities use list-prefix states.
- Conditional Reward Estimation: Reward heads estimate conditional item rewards only at reached positions, using sigmoid targets for binary engagement and bucketized distributions for continuous play time.The predicted rewards can combine multiple task-specific heads with weights.
- List Scoring: The evaluator combines estimated survival probabilities and conditional rewards into a session-wise current-list contribution used to compare candidate lists.For online re-ranking, the boundary transition is fixed to 1 because the evaluator runs after the request has been triggered.
- Training: Training uses survival-probability and conditional-reward losses that directly supervise SWIM’s boundary-augmented survival chain and reached-position rewards.These are the two main components of the overall training objective.
5 Theoretical Results
SWIM’s boundary-augmented survival parameterization represents session-wise list contribution by incorporating whether the current request is reached from the preceding prefix. Its distribution-free discrete survival formulation is more expressive than Weibull-restricted modeling for irregular finite-horizon continuation patterns.
- 5.1 Session-prefix Advantage: SWIM models session-wise list contribution using a boundary transition from the preceding session prefix and in-list continuation probabilities.The boundary transition captures whether the request is reached, while subsequent transitions model continuation across positions.
- 5.1 Session-prefix Advantage: Request-only evaluators incur strictly positive irreducible error when session-wise contribution varies across prefixes sharing the same request-level information.The minimum squared-loss risk is the conditional variance of the session-wise target given request-level information.
- 5.1 Session-prefix Advantage: SWIM conditions the boundary transition on preceding session-prefix information, allowing it to model signals varying with browsing history, accumulated engagement, and user fatigue.These signals distinguish session-wise evaluation from request-level evaluation.
- 5.2 Prior-free Survival Modeling: SWIM directly parameterizes discrete continuation probabilities, whereas CAVE uses a two-dimensional Weibull survival manifold with scale and shape parameters.The Weibull prior can provide useful regularization when stochastic exit behavior is smooth and Weibull-like.
- 5.2 Prior-free Survival Modeling: Early survival mismatches can affect estimated contributions at all downstream positions because local residuals are weighted by downstream residual value.Non-Weibull continuation patterns therefore create propagation effects in list evaluation under the restricted model.
- 5.2 Prior-free Survival Modeling: For finite horizons, SWIM can represent any valid discrete survival profile, including position-irregular drops, contextual disruptions, repeated content, and session-stage-specific fatigue.The full discrete continuation space avoids the structural restriction of the lower-dimensional Weibull family.
6 Experiments
Experiments evaluate SWIM across public datasets and Kuaishou traffic using common re-ranking protocols, offline metrics, ablations, and online engagement measures. SWIM achieves the best offline performance on both datasets, while component analyses and online testing assess its design and practical impact.
- Datasets: SWIM is evaluated on RecFlow and KuaiRand offline and on Kuaishou’s real traffic online.RecFlow provides upstream candidate sets, while KuaiRand requires reconstructed request chunks; Kuaishou logs contain production candidate sets, generated lists, positions, prefixes, rewards, and serving data.
- Experimental setup: Experiments compare SWIM with pointwise, listwise, G-only, and G-E re-ranking baselines under a shared temporal evaluation protocol.Training uses the earliest 90% of requests and testing uses the latest 10%; G-E methods generate 20 candidate lists before evaluator selection.
- Evaluation metrics: Offline metrics use video-view feedback: NDCG(@6) measures ordered-list quality, while AUC measures discrimination between video-view-positive and video-view-negative candidates.Both metrics are computed with the binary video-view label; NDCG evaluates six ranked positions.
- Main results: SWIM achieves the best performance on both datasets and both offline metrics, outperforming pointwise, listwise, G-only, and G-E baselines.The reported results attribute this advantage to modeling user continuation and reached-position conditional reward.
- Ablation study: Ablations show that prefix conditioning, boundary transition modeling, and recursive survival weighting each contribute to final performance.The components respectively support session-aware representation, prefix-to-request survival supervision, and reachability-aware reward weighting.
7 Conclusion
SWIM evaluates generative re-ranking lists through a boundary-augmented survival process that connects session prefixes with step-wise continuation and rewards. Future work targets counterfactual evaluation and longer, more heterogeneous browsing trajectories.
- SWIM models each candidate list with a boundary-augmented survival process spanning the session prefix and within-list continuation behavior.This formulation combines recursive survival probabilities with reached-position conditional rewards.
- SWIM estimates the prefix-conditioned contribution of the current list to the session-level objective by combining survival probabilities and conditional rewards.
- Future work should develop counterfactual and off-policy evaluation protocols for generative list spaces.
- Future work should extend session-prefix modeling to longer and more heterogeneous browsing trajectories.
GenAI Usage Disclosure
Generative AI was used only for editorial assistance and not for content generation or substantive intellectual contributions.
- Generative AI was used for grammar checking, sentence refinement, and improving conciseness.
- Generative AI was not used to generate content or make substantive intellectual contributions.