Source-linked AI summary
OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking System
Sunhao Dai, Jiakai Tang, Jiahua Wu, Kun Wang, Yuxuan Zhu, Bingjun Chen, Bangyang Hong, Yu Zhao, Cong Fu, Kangle Wu, Yabo Ni, Anxiang Zeng, Wenjie Wang, Xu Chen, Jun Xu, See-Kiong Ng
TL;DR
Industrial ranking systems have largely transplanted Transformer architectures, leaving LLM-style context engineering and multi-step reasoning underexplored. OnePiece integrates structured context, block-wise latent reasoning, and progressive multi-task training across retrieval and ranking, achieving over +2% GMV/UU improvement and +2.90% advertising revenue improvement in Shopee’s personalized search deployment.
Problem
LLM-style context engineering and multi-step reasoning remain largely underexplored in industrial ranking systems beyond incremental Transformer-based improvements over strong DLRMs.
Method
OnePiece uses structured context tokens, block-wise latent reasoning, and progressive multi-task training with feedback chains across retrieval and ranking.
Results
+2.90% advertising revenue and over +2% GMV/UU improvement were reported in Shopee’s online personalized-search deployment.
Takeaways & Limitations
The results support OnePiece as a practical unified framework for scalable reasoning-driven industrial ranking.
Takeaways & Limitations
The production ranking model uses a downgraded variant with block size M=1 and substantially fewer item side-information features.
Abstract
from arXiv · showhide
Despite the growing interest in replicating the scaled success of large language models (LLMs) in industrial search and recommender systems, most existing industrial efforts remain limited to transplanting Transformer architectures, which bring only incremental improvements over strong Deep Learning Recommendation Models (DLRMs). From a first principle perspective, the breakthroughs of LLMs stem not only from their architectures but also from two complementary mechanisms: context engineering, which enriches raw input queries with contextual cues to better elicit model capabilities, and multi-step reasoning, which iteratively refines model outputs through intermediate reasoning paths. However, these two mechanisms and their potential to unlock substantial improvements remain largely underexplored in industrial ranking systems. In this paper, we propose OnePiece, a unified framework that seamlessly integrates LLM-style context engineering and reasoning into both retrieval and ranking models of industrial cascaded pipelines. OnePiece is built on a pure Transformer backbone and further introduces three key innovations: (1) structured context engineering, which augments interaction history with preference and scenario signals and unifies them into a structured tokenized input sequence for both retrieval and ranking; (2) block-wise latent reasoning, which equips the model with multi-step refinement of representations and scales reasoning bandwidth via block size; (3) progressive multi-task training, which leverages user feedback chains to effectively supervise reasoning steps during training. OnePiece has been deployed in the main personalized search scenario of Shopee and achieves consistent online gains across different key business metrics, including over $+2\%$ GMV/UU and a $+2.90\%$ increase in advertising revenue.
1. Introduction
Industrial ranking systems have mostly transplanted Transformer architectures, while LLM-inspired context engineering and multi-step reasoning remain underexplored. OnePiece addresses this gap across retrieval and ranking, with offline and online evaluations showing consistent gains.
- Existing industrial ranking efforts generally focus on Transformer-based sequential architectures, whose improvements over strong DLRMs remain incremental.
- LLM-style context engineering enriches inputs, while multi-step reasoning iteratively refines outputs through intermediate reasoning steps.
- The framework combines structured context engineering, block-wise latent reasoning, and progressive multi-task training.These components target richer inputs, iterative representation refinement, and supervision from user feedback chains.
- +2.90% advertising revenue and over +2% GMV/UU improvement were achieved in Shopee’s personalized search deployment.The reported online results include retrieval and ranking gains across business metrics.
- OnePiece is presented as the first industrial-scale ranking framework to deploy context engineering and multi-step reasoning across retrieval and ranking.
2. Preliminary
Industrial cascade ranking uses multiple stages to balance efficiency and ranking quality. Retrieval performs coarse candidate selection, while ranking jointly models retrieved candidates for fine-grained scoring.
- The ranking objective is to order items by their predicted likelihood of receiving the user’s next interaction.
- Cascade ranking filters massive item pools with lightweight early stages before applying more sophisticated models to fewer candidates.
- Retrieval Stage: Retrieval selects a small candidate set using disjoint user and item encoders with approximate nearest-neighbor search.
- Ranking Stage: Ranking jointly encodes each retrieved candidate with full user context and predicts precise preference scores using a Transformer backbone and MLP head.
- Retrieval targets coarse-grained recall over the full corpus, whereas ranking performs fine-grained discrimination among retrieved candidates.
3. The Proposed OnePiece Framework
OnePiece unifies retrieval and ranking with structured tokenized context, block-wise latent reasoning, and progressive multi-task training on a pure Transformer backbone. Its context combines behavioral, preference, situational, and candidate information while grouped setwise ranking balances comparison with tractable cost.
- 3.1. Overview: OnePiece combines structured context engineering, block-wise latent reasoning, and progressive multi-task training across industrial retrieval and ranking.
- 3.1. Overview: Block-wise reasoning iteratively refines hidden representations, increasing reasoning bandwidth through multiple reasoning steps.
- 3.2. Context Engineering: Interaction history, preference anchors, and situational descriptors are shared across modes, while ranking additionally includes candidate item set tokens.Candidate tokens enable joint candidate scoring in the single-tower ranking architecture.
- 3.2. Context Engineering: Preference anchors inject domain-knowledge reference signals, such as top-clicked items, to guide predictions toward plausible preference representations.
- 3.2. Context Engineering: Grouped setwise ranking partitions retrieved candidates into groups such as 12 items, enabling intra-group comparison while keeping reasoning cost tractable.The strategy lies between pointwise ranking and full-set ranking.
3.3. Backbone Architecture
OnePiece uses a unified Transformer encoder for retrieval and ranking, then performs block-wise latent refinement with causal information flow and progressive supervision. Task-specific block sizes and objectives expose intermediate representations for retrieval and ranking learning.
- Transformer-Based Sequential Encoding: A bidirectional Transformer encodes the packed input sequence, whose final hidden states provide the foundation for subsequent reasoning.The backbone uses pre-normalization, multi-head self-attention, and feed-forward layers; bidirectional attention lets tokens use full non-autoregressive context.
- Block-Wise Multi-Step Reasoning: Block-wise reasoning iteratively refines hidden-state blocks, with current blocks attending to input and earlier blocks but not future blocks.Reasoning position embeddings distinguish steps, while the causal block-wise mask constrains information flow.
- Block-Wise Multi-Step Reasoning: Block size M controls reasoning bandwidth and is selected by task, including situation-descriptor length for retrieval and candidate-group size for ranking.Retrieval uses aggregation blocks for user and query signals, while ranking uses M = C so each block contains candidate states.
- Progressive Multi-Task Training: Progressive multi-task training assigns increasingly complex behavioral objectives to successive reasoning steps, providing process supervision for intermediate representations.The curriculum can follow exposure → click → purchase, with each block optimizing its designated task.
- Retrieval and Ranking Objectives: Retrieval combines calibrated point-wise estimation with bidirectional contrastive learning, whereas ranking combines binary cross-entropy with set-level contrastive learning across reasoning steps.Retrieval optimizes user-item and item-user contrastive objectives; ranking trains candidate-wise scores with complementary point- and set-wise losses.
3.5. Time Complexity Analysis
The complexity analysis separates the backbone cost from the incremental reasoning cost. KV caching limits each reasoning step to computation for the new block and attention against cached tokens.
- Backbone Complexity: The backbone encoder costs O(L(N^2d + Nd^2)) across L Transformer layers for sequence length N and hidden dimension d.This accounts for self-attention and feed-forward computation without reasoning.
- Reasoning Complexity: KV caching makes each reasoning step process only M new tokens, with per-layer work for projections and attention to N + (k−1)M cached tokens.At step k, the stated components are O(Md^2), O(M(N + kM)d), and O(Md^2).
- Reasoning Complexity: Total reasoning overhead is aggregated across K reasoning steps and L Transformer layers.The analysis presents the additional cost separately from the backbone encoder cost.
4. Offline Experiments
Offline experiments evaluate OnePiece on Shopee data using streaming-style training, retrieval recall, and ranking AUC/GAUC across multiple feedback types. OnePiece achieves the strongest overall results, while ablations show complementary gains from structured context, block-wise reasoning, and progressive multi-task training.
- Overall Performance: Recall@100 rises from 0.485 to 0.517 and C-AUC from 0.862 to 0.911 over the strongest ReaRec+PA baseline, giving OnePiece the best overall results.The gains are attributed to block-wise latent reasoning and progressive multi-task training.
- Context Engineering Ablation: Adding situational descriptors raises Recall@100 from 0.504 to 0.517, whereas ranking gains remain marginal because interaction history and preference anchors already provide rich signals.The result indicates that situational descriptors contribute stronger contextual grounding during retrieval than during candidate ranking.
- Context Engineering Ablation: Structured context improves performance by combining interaction history, scalable query-specific preference anchors, and situational descriptors.Preference anchors provide complementary query-dependent signals, while situational descriptors particularly benefit retrieval.
- Training Strategy Ablation: Block-wise reasoning produces cumulative gains as reasoning depth increases, reaching retrieval R@100 of 0.510 and ranking C-AUC of 0.906.The reported progression includes R@100 improvements from 0.470 to 0.490 and then 0.495 before deeper reasoning reaches 0.510.
- Training Strategy Ablation: Progressive multi-task training improves retrieval R@100 from 0.510 to 0.517 and ranking C-AUC from 0.906 to 0.911 over single-embedding supervision.Different reasoning steps specialize in task-specific information, with retrieval favoring two steps and ranking three.
- Data Efficiency: OnePiece surpasses DLRM and HSTU after 7–10 training days and scales more effectively as additional data becomes available.Training curves show smooth growth for OnePiece, while DLRM quickly plateaus.
5. Online A/B Testing
Online A/B tests show that OnePiece improves retrieval coverage, business outcomes, and ranking quality, while maintaining favorable computational scaling for industrial deployment.
- Retrieval Mode: +1.08% GMV/UU and +0.66% CTCVR in retrieval, alongside lower Bad Query Rate and gains across transaction metrics.GMV(99.5%)/UU rises +0.91%, Order/UU +0.71%, Paid Order/UU +0.98%, and Buyer +0.41%.
- Ranking Mode: +1.12% GMV/UU and +2.9% AR/UU in ranking, with +0.29% CTR but +0.21% Bad Query Rate.Order/UU and Buyer each increase by +0.08%, indicating a trade-off between revenue gains and query relevance.
- Recall Coverage and Exclusive Contribution: OnePiece improves recall coverage across routes, including STR1 from 37.3% to 66.2% and STR2 from 31.3% to 64.4%.The comparison uses U2I exposure overlap with other recall strategies.
- Recall Coverage and Exclusive Contribution: Exclusive impression share rises from 3.6% to 9.9%, while exclusive click share increases from 2.4% to 5.7%.These results indicate substantial unique contribution beyond overlap with other recall routes.
- Efficiency Analysis: Retrieval inference time falls 25% from 40ms to 30ms, while ranking latency rises from 110ms at M=1 to 120ms at M=12.The ranking increase is 10.1% for a 12× reasoning-capacity expansion, while C-AUC improves from 0.885 to 0.927.
6. Related Work
Related work has largely adapted Transformer architectures or explored isolated LLM-inspired techniques, leaving a principled integration of context engineering and reasoning underdeveloped for ranking systems.
- Context Engineering and Reasoning Enhancement in LLMs: LLM research frames context engineering and reasoning enhancement as complementary pathways for improving model utility.Context engineering organizes prompts, knowledge, memory, and histories, while reasoning enhancement advances computational reasoning.
- Context Engineering and Reasoning Enhancement in Ranking Systems: Ranking research has progressively adopted language-modeling advances, but early work mainly focused on directly applying Transformer-based models.Recent efforts mirror LLM research directions but do not fully establish their underlying design principles.
- Context Engineering and Reasoning Enhancement in Ranking Systems: OnePiece addresses the stated gap by establishing a unified architectural paradigm that adapts LLM design principles to ranking systems.The framework integrates structured context engineering and block-wise latent reasoning rather than treating them as isolated techniques.
7. Conclusion and Future Directions
OnePiece unifies structured context engineering, latent reasoning, and progressive multi-task training across industrial retrieval and ranking. Its online deployment shows consistent business gains and motivates unified multi-route retrieval as a future direction.
- Conclusion: OnePiece combines structured context engineering, block-wise latent reasoning, and progressive multi-task training in a pure Transformer framework.The method organizes heterogeneous signals into a structured tokenized sequence and supervises reasoning with user feedback chains.
- Unified Multi-Route Retrieval: Figure 7 contrasts separate route-specific models with OnePiece’s single model using tailored prompts for different retrieval scenarios.The unified design is presented as reducing system complexity and maintenance overhead.
- Conclusion: Online A/B testing reports over +2% GMV/UU and +2.90% advertising revenue improvement, alongside superior efficiency and hardware utilization.These gains were observed in Shopee’s main personalized search scenario.
- Future Directions: Future work highlights unified multi-route retrieval, while noting inherent limitations in scaling latent reasoning.The proposed direction uses scenario-specific context engineering to serve diverse recommendation objectives.
A. Offline Dataset Construction
The dataset construction uses stage-specific interaction signals: retrieval learns impression and click objectives, whereas ranking refines preferences across the conversion funnel.
- Retrieval Stage: Retrieval samples use clicked items as positives for impression and click tasks after filtering sessions without clicks.Exposed but unclicked items are also incorporated into retrieval sample construction.
- Ranking Stage: Ranking samples use task-specific interactions such as impression, click, add-to-cart, and order as positives.Interactions from preceding conversion-funnel tasks serve as negatives for downstream ranking objectives.
B. Context Engineering Details
OnePiece constructs a unified tokenized context from interaction history, preference anchors, situational descriptors, and candidate items. These components encode sequential behavior, query-associated preferences, user context, and item features for retrieval and ranking.
- Context components: OnePiece unifies interaction history, preference anchors, situational descriptors, and candidate items into structured input components.Candidate Item Set uses straightforward item features, while the other components receive specialized construction procedures.
- Interaction History: Interaction History merges users’ recent click, cart, and purchase sequences chronologically, with each item represented by identity, category, shop, and related attributes.The history covers the most recent month.
- Preference Anchor: Preference Anchor aggregates query-associated top-k exposures, clicks, and purchases, separating sequence types with [BOS] and [EOS] tokens.Unlike Interaction History, Preference Anchor uses sequence concatenation rather than mixing.
- Situational Descriptors: Situational Descriptors encode user profiles and query context through User and Query tokens containing demographic, identifier, textual, popularity, and related features.The User Token integrates attributes such as user ID, age, and location, while the Query Token incorporates query ID and textual content.
C. Attention Visualization Analysis
Attention visualizations show structured, increasingly integrated attention across OnePiece’s context components and progressive refinement across reasoning blocks. Retrieval emphasizes compact preference and situational interactions, while ranking expands attention to jointly evaluate candidates.
- Context attention: Layer-1 heads emphasize localized or diagonal attention, whereas Layer-2 heads connect multiple token groups for broader integration across modes.The visualizations also show complementary head-level strategies within layers.
- Retrieval mode: Retrieval attention remains relatively localized, with Layer-2 heads linking interaction history to preference anchors and situational descriptors to interaction history.These patterns align with retrieval’s coarse-grained candidate-filtering objective.
- Ranking mode: Ranking attention spans interaction history, preference anchors, situational descriptors, and candidate items simultaneously for joint preference–candidate evaluation.Adding candidate-item tokens expands the attention space to four-way interactions.
- Block-wise reasoning: In retrieval reasoning, R1 concentrates on situational descriptors and preference anchors while giving minimal attention to interaction history.The initial reasoning step prioritizes contextual and query-specific signals.
- Block-wise reasoning: Ranking reasoning progressively consolidates earlier outputs, with R3 attending strongly to R2 and more weakly to R1.The three-step process follows R1 → R2 → R3 and exhibits hierarchical information enhancement.