PaperlayerView exact revision on arXiv

Source-linked AI summary

OneRank: Unified Transformer-Native Ranking Architecture for Multi-Task Recommendation

Jiakai Tang, Sunhao Dai, Kun Wang, Zhiluohan Guo, Yu Zhao, Cong Fu, Kangle Wu, Yabo Ni, Anxiang Zeng, Xu Chen, Jun Xu

arXiv:2606.16838v1cs.IR

TL;DR

Existing Transformer-based multi-task recommenders retain an encoder–predictor split that limits task-specific representation learning and creates gradient interference. OneRank internalizes multi-task reasoning in the Transformer with task-private channels and dynamic matching-based scoring, and experiments show significant gains over state-of-the-art baselines.

  • Problem

    Transformer-based multi-task recommenders largely retain an encoder–predictor split, creating a task-agnostic information bottleneck and exposing shared parameters to conflicting task gradients.

  • Method

    OneRank internalizes multi-task reasoning within the Transformer through task-private channels, controlled cross-task attention with gradient detachment, and dynamic matching-based ranking.

  • Results

    Offline and online experiments on large-scale industrial datasets show that OneRank significantly outperforms state-of-the-art baselines while maintaining computational efficiency.

  • Takeaways & Limitations

    OneRank provides a unified Transformer-native architecture for task-specialized, context-aware multi-task ranking with controlled knowledge sharing.

Abstract

from arXiv · show

Multi-task learning (MTL) is essential in recommender systems to enable complementary learning among diverse user feedback. While modern industrial practices have shifted from DNNs to Transformer-centric architectures to strengthen sequence modeling and scaling capacity, they still decouple feature encoding from multi-task prediction, treating the Transformer as a task-agnostic encoder. This design fundamentally limits the performance and scalability by (1) creating an information bottleneck under heterogeneous task objectives, (2) inducing gradient interference that leads to the seesaw phenomenon, and (3) forcing a dataflow transition in which attention-based, context-adaptive representation learning is converted to static feed-forward task prediction with incompatible information read-write dynamics. We propose OneRank, a Transformer-native multi-task ranking framework that eliminates encoder-predictor separation and introduces task-private channels for forward representation learning and backward optimization, enabling task-specialized learning while reducing inter-task interference. In the forward pass, OneRank learns task-specific representations bottom-up through task-conditioned information selection, candidate-aware contextualization, and controlled cross-task interaction. In the backward pass, cross-task gradient detachment isolates task-private parameter updates from shared knowledge extraction modules, preventing negative transfer. We further replace static task-specific MLP scorers with dynamic matching-based scoring for context-aware personalized ranking. By internalizing multi-task reasoning within the Transformer stack, OneRank establishes a unified and scalable architectural paradigm. Offline and online experiments on large-scale industrial datasets show that OneRank significantly outperforms state-of-the-art baselines while maintaining computational efficiency.

1 Introduction

OneRank replaces the conventional encoder–predictor separation in Transformer-based multi-task recommendation with a unified Transformer-native architecture. It learns task-specialized representations and rankings through task-private channels, contextualized information routing, controlled cross-task interaction, and gradient detachment.

  • Architectural limitations: Existing recommender systems largely retain an encoder–predictor design that maps inputs to a shared, task-agnostic representation before task-specific prediction.This structure is formalized as G(Z = F(X)), where F produces shared representation Z and G contains task-specific predictors.
  • Architectural limitations: The shared representation creates a task-agnostic information bottleneck by entangling task-specific signals with shared knowledge, leaving predictors to disentangle them.Increasing Transformer encoding capacity does not remove this structural constraint.
  • Architectural limitations: Shared-bottom architectures exhibit the seesaw phenomenon because conflicting task gradients update shared parameters without separating task-specific optimization directions.Such interference can improve one task while degrading others.
  • Architectural limitations: Encoder–predictor separation mismatches Transformers’ context-dependent information routing with static feed-forward predictors that have limited ability to adapt to dynamic user context.This transition disrupts end-to-end task-aware learning.
  • OneRank framework: OneRank internalizes multi-task reasoning within the Transformer stack using bottom-up task-private channels, candidate-aware contextualization, controlled cross-task interaction, and strategic gradient detachment.Task-specific tokens support early specialization, while relational attention selectively injects task dependencies when beneficial.

2 Methodology

OneRank internalizes multi-task reasoning within the Transformer, replacing the conventional encoder–predictor split with task-private channels alongside shared pathways. Its methodology combines unified tokenization, input-level task specialization through mutual invisibility, candidate-aware contextualization, and dynamic scoring.

  • Architecture: OneRank eliminates encoder–predictor separation by integrating task-private channels with task-shared pathways throughout the Transformer.This bottom-up design targets task specialization while preserving beneficial knowledge sharing across architectural levels.
  • Input representation: The methodology structures heterogeneous inputs into unified token sequences, combining sequential patterns and feature interactions for joint modeling.Inputs include user interaction history, preference anchors, and candidate-task token groups.
  • Input representation: Candidate-task groups separate shared user context from task-private channels, while structured attention masking enables independent candidate-specific task representations and parallel computation.Task tokens attend to candidate-specific information and shared user context while groups operate independently during encoding.
  • Task specialization: Task-specific token injection with mutual invisibility enables early task specialization and mitigates the seesaw phenomenon by preventing task tokens from attending to one another.The mask preserves causal visibility in user context while isolating candidate groups and task tokens.
  • Scoring: The Transformer stack applies masked Multi-Head Self-Attention with residual connections and layer-based encoding before matching-based dynamic scoring.This sequence completes OneRank’s Transformer-native ranking pipeline.

Q Q MHCA MHCA ①②

OneRank uses task-specific multi-head cross-attention to aggregate candidate-aware global information, preserving separate aggregation pathways while modeling cross-candidate competition. Configurable cross-task attention then enables selective forward knowledge transfer with diagonal-only backward gradients, followed by dynamic matching-based scoring for context-aware ranking.

  • Candidate-Aware Contextualization: Situational descriptors anchor aggregation with user demographics, query information, and session metadata such as time and location.These descriptors provide contextual signals for candidate-aware aggregation.
  • Candidate-Aware Contextualization: Task-specific MHCA aggregates information across the candidate set through independent parameterized pathways, capturing cross-candidate competitive dynamics.Each task maintains its own aggregation flow while producing a task-wise global representation.
  • Cross-Task Attention: A configurable cross-task attention mask determines which tasks each task can attend to, supporting domain-specific information-flow patterns.The mechanism accommodates relationships ranging from strict cascades to fully autonomous task interactions.
  • Cross-Task Attention: Diagonal-only gradient flow blocks off-diagonal task gradients during backpropagation, mitigating inter-task conflicts while preserving beneficial forward transfer.Attended tasks act as read-only memory for knowledge transfer, so optimizing one task does not adversely affect others.
  • Dynamic Matching-Based Scoring: Dynamic matching scores task-candidate relevance through inner-product similarity between task-aware global context and context-conditioned candidate embeddings.This replaces fixed context-independent MLP transformations with session-adaptive, task-adaptive ranking.

3 Discussion

OneRank addresses the training-serving mismatch and seesaw phenomenon through context-aware candidate-set modeling, dynamic scoring, and multi-level task decoupling. Its unified Transformer design also supports flexible task dependencies and efficient scalable ranking.

  • Context-aware ranking: OneRank models entire candidate sets with cross-attention, capturing competitive dynamics, relative preferences, and candidate-pool distributions rather than isolated absolute scores.This addresses the mismatch between point-wise training and serving-time ranking of candidate sets.
  • Context-aware ranking: Matching-based scoring uses contextual task representations, allowing the same user-item pair to receive different scores across sessions and jointly optimizing representations in a shared geometric space.The contextual representation incorporates session-specific intent, query semantics, and temporal context; the formulation supports semantic alignment and improved gradient flow.
  • Task decoupling: OneRank mitigates seesaw effects through input-, intermediate-, and prediction-level decoupling, isolating task-specific extraction, aggregation, and backward optimization while retaining forward knowledge transfer.Gradient detachment permits other tasks’ representations to benefit a task during the forward pass while restricting cross-task gradient flow during optimization.
  • Flexible task dependencies: Configurable cross-task attention masks encode strict cascades, bidirectional attention, or hybrid dependencies within one architecture, avoiding architecture search and task-specific model variants.The framework is contrasted with ESMM’s fixed cascades and MMoE’s independent towers.
  • Efficiency and scalability: The unified Transformer-native architecture removes the F-G transition, reduces redundant context encoding through single-user multiple-candidate training, and enables efficient serving with KV-caching.The discussion presents this co-design as improving modeling capacity, optimization stability, computational efficiency, and adaptability for industrial multi-task ranking.

4 Offline Evaluation

Offline experiments on Shopee’s large-scale interaction dataset evaluate OneRank across encoder architectures and multi-task learning strategies. OneRank achieves the best performance across all tasks and metrics, while ablations and scaling studies validate task specialization and the scalability of its Transformer-native design.

  • Experimental Setup: The offline evaluation uses 30 consecutive days of Shopee logs covering click, add-to-cart, and order feedback, reporting AUC and GAUC for each task.The dataset is collected from a large-scale proprietary e-commerce platform.
  • Experimental Setup: The comparison disentangles encoder capacity from multi-task optimization by evaluating combinations of DNN, Transformer-based encoders, and multiple multi-task learning strategies.The design compares OneRank against encoder and optimization alternatives under consistent evaluation settings.
  • Overall Results: Multi-task learning consistently improves click, add-to-cart, and order prediction over noMTL when using conventional DNN encoders.The result confirms complementary supervision from jointly modeling dense and sparse user feedback.
  • Overall Results: Transformer-based encoders such as MTGR and OneTrans further improve performance across most multi-task strategies, whereas DCMT performs poorly under Transformer encoders.The paper attributes the DCMT result to possible over-correction of sparse tasks by its debiasing-oriented design.
  • Overall Results: OneRank outperforms all baseline combinations across every reported metric and task while using compact parameterization and a moderate computation increase.The result is attributed to unifying representation learning and multi-task ranking within the Transformer architecture.
  • Ablation Studies: Removing task-specific tokens reduces A-AUC from 0.8463 to 0.8424 and O-GAUC from 0.8350 to 0.8337, validating early task specialization.A single shared token also underperforms the full model, while removing cross-task attention produces mixed results compared with the shared-token variant.

5 Related Work

Prior work in recommendation spans multi-task learning based on structured knowledge transfer and Transformer-based ranking architectures. OneRank differs by internalizing multi-task reasoning within a unified Transformer through task-specific tokens, candidate-aware contextualization, gradient detachment, and dynamic matching-based scoring.

  • Multi-Task Learning for Recommendation: Multi-task recommendation methods jointly model diverse user behaviors, with ESMM and ESCM exemplifying structured dependency modeling through conditional relationships and counterfactual reasoning.ESMM addresses data sparsity through the CTR–CVR conditional relationship, while ESCM further refines this approach with counterfactual reasoning.
  • Transformer-Based Ranking: Transformer-based ranking research addresses long-sequence and heterogeneous-sequence modeling, while unified architectures jointly model feature interactions and sequential patterns within one Transformer.The passage identifies KuaiFormer for long-sequence modeling, Climber for heterogeneous sequences, and HHFT, MTGR, OneTrans, and HyFormer as unified architectures.
  • OneRank: OneRank internalizes multi-task reasoning through task-specific tokens with mutual invisibility, candidate-aware contextualization, strategic gradient detachment, and dynamic matching-based scoring.The passage characterizes these mechanisms as enabling superior scaling and stable multi-task optimization.

6 Conclusion

The conclusion identifies encoder–predictor separation as a source of bottlenecks, gradient conflicts, and inflexible ranking architectures in multitask recommender systems. It presents OneRank as a unified Transformer-native framework designed to address these limitations by internalizing multi-task reasoning.

  • 6 Conclusion: Existing multitask recommender systems suffer from task-agnostic information bottlenecks, gradient conflicts causing the seesaw phenomenon, and architectural transitions that hinder context-aware dynamic ranking and scaling.These limitations are attributed to conventional encoder-predictor separation.
  • 6 Conclusion: OneRank is proposed as a unified Transformer-native multi-task ranking framework to address the limitations of conventional encoder-predictor separation.The framework is intended to unify representation learning and multi-task reasoning within the ranking architecture.
  • 6 Conclusion: OneRank internalizes multi-task reasoning within a unified architecture rather than preserving the conventional encoder-predictor separation.This architectural direction directly targets the identified information, optimization, and dynamic-ranking limitations.

A Offline Experimental Setup … B.1 Online Deployment Details

The paper evaluates OneRank offline on a proprietary Shopee dataset using feedback-specific AUC and GAUC metrics, then deploys it online through score fusion and parallel cross-attention ranking. The setup targets realistic multi-task ranking with business-objective balancing and large candidate pools.

  • A.1 Dataset Details: Offline experiments use a large-scale proprietary dataset collected from Shopee, built from 30 consecutive days of user interaction logs.The dataset addresses the lack of public data combining rich sequential item features, explicit multi-task annotations, and ranking-stage candidate sets.
  • A.2 Evaluation Metrics: Model performance is assessed using click, add-to-cart, and order feedback signals.These signals are denoted C, A, and O, respectively.
  • A.2 Evaluation Metrics: For each feedback signal, the evaluation reports both AUC and GAUC as C-AUC/C-GAUC, A-AUC/A-GAUC, and O-AUC/O-GAUC.The reporting protocol follows prior work cited in the paper.
  • B.1 Online Deployment Details: OneRank is deployed in Shopee’s standard multi-stage ranking pipeline through score fusion of multiple task outputs into a unified ranking score.The production score combines predicted click-through rate, conversion rate, price, advertising revenue, and relevance terms.
  • B.1 Online Deployment Details: The fused score balances GMV optimization, advertising revenue, search relevance, and user intent alignment through tuned coefficients a, b, and c.The first term uses pctr, pcvr, and price; the second uses pctr and ecpm; and the third uses relevance.
  • B.1 Online Deployment Details: Online requests with up to 4,096 candidate items are partitioned into 8×512 groups scored in parallel.Each group independently performs cross-attention-based ranking, scaling OneRank to large candidate pools while preserving context-aware list modeling.

B.2 Evaluation Protocol and Metrics

The online evaluation uses a 7-day A/B test with equal 10% treatment and control traffic allocations, measuring platform benefits and user experience through complementary metrics.

  • Evaluation Protocol: Online A/B testing runs for 7 days from January 8 to January 14, 2026, with 10% live traffic assigned to OneRank and 10% to baseline control.The protocol follows standard industrial evaluation practices.
  • Metrics: Platform benefits are measured by GMV/UU, Paid Orders/UU, and AR/UU.These represent gross merchandise value per user, completed paid orders per user excluding refunds, and advertising revenue per user, respectively.
  • Metrics: User experience is measured by Bad Query Rate, the proportion of user queries judged irrelevant and a proxy for recommendation accuracy and user satisfaction.

C Parameter Sensitivity Analysis

OneRank is most sensitive to the contrastive-loss temperature and the relative InfoNCE/BCE weighting. Performance peaks near temperature 0.2, while equal loss weights provide the best results across tasks and metrics.

  • Temperature Sensitivity: Performance improves as temperature decreases, peaks around 0.2, and slightly declines when temperature becomes too small.The trend holds across AUC and GAUC for click, add-to-cart, and order prediction.
  • Temperature Sensitivity: The temperature trend is consistent with contrastive learning because large temperatures overly smooth similarity distributions and weaken discrimination.The supplied passage introduces this explanation but is truncated before completing the mechanism.
  • Loss-Weight Sensitivity: Equal weights for InfoNCE and BCE consistently achieve the best results across all tasks and evaluation metrics.Table 4 compares different loss-weight ratios, while over-weighting BCE at 1:2 degrades performance.
Loading 2606.16838v1…