PaperlayerView exact revision on arXiv

Source-linked AI summary

DREAM Technical Report

Bin Zhang, Bowen Zheng, Chao Yi, Chengyu Lai, Dian Chen, Dimin Wang, Gaoyang Guo, Jialin Zhu, Jian Wu, Jing Yu, Jiuning Lin, Lingqing Zhang, Lingyun Zheng, Mao Zhang, Mingming Pan, Ruiquan Lan, Shuai Zhong, Wen Chen, Wendong Zhang, Xiaodong Zhu, Xuan Chen, Xunke Xi, Yifan Lu, Yiheng Wang, Yue Zeng, Yujie Luo, Yuning Jiang, Zhe Hu, Zhibo Xiao, Zihong Huang, Binbin Cao, Bo Zheng, Danning Wang, Dixuan Wang, Ge Fan, Haixia Wu, Han Zhu, Hao Fang, Haoming Chen, Huiping Chu, Jian Wang, Jianjun Wu, Jiawei Wu, Jiaxin Yu, Jingwen Liu, Jinzhe Shan, Kai Meng, Kai Zhang, Keqin Xu, Kewei Zhu, Lang Tian, Leihui Chen, Li Chen, Licheng Xu, Lide Xiao, Ruitong Zhang, Shiyao Peng, Silu Zhou, Tao Wang, Wei Shi, Wenjun Yang, Xiang Chen, Xiang Gao, Xiao Ren, Xu Liu, Xuwen Wang, Yang Li, Yeqiu Yang, Yi Hu, Yichen Yuan, Yinnan Song, Yipeng Yu, Yuan Liu, Yunqi Gao, Zhiliang Huang, Zhujin Gao, Zongyuan Wu

arXiv:2608.09408v3cs.IR

TL;DR

Industrial recommendation pipelines struggle with fragmented information and objectives, rigid strategies, and limited real-time intent awareness. DREAM adds an intent-aware policy layer over existing pipelines, combining structured perception, layered strategy control, and offline-online feedback; Taobao A/B tests report larger gains when control extends from re-ranking to fine ranking.

  • Problem

    Industrial recommender pipelines fragment information and objectives, rely on rigid strategies, and inadequately address session-level intent shifts.

  • Method

    DREAM overlays existing pipelines with an Intent Engine, a layered Meta Engine, unified guarded dispatch, and a Reward Dual Loop combining offline simulation with online feedback.

  • Results

    2.06% IPV, 2.39% Core IPV, and 0.88% GMV gains from re-ranking control increase to 2.71%, 3.06%, and 1.31% with fine-ranking control, while PV consistently rises by more than 1%.

  • Takeaways & Limitations

    DREAM supports agentic meta-control as a practical industrial recommendation paradigm without replacing pipeline models or compromising serving stability.

  • Takeaways & Limitations

    Replay conclusions remain relative to the production environment used during the replay window because mean aggregation does not exactly reconstruct historical serving state.

Abstract

from arXiv · show

Industrial recommender systems commonly use cascaded retrieval, ranking, and re-ranking pipelines. Although efficient, these pipelines fragment information and objectives across modules, rely on rigid rules, and have limited awareness of real-time intent, leaving session-level shifts among browsing, comparison, and purchase insufficiently addressed. We present DREAM (Developing Recommender Engine with Agentic Methods), an autonomous optimization control architecture that adds a perception-aware, orchestrable, and auditable policy layer atop existing pipelines without replacing them. DREAM has two core components. First, a three-tier Intent Engine fuses on-device signals into structured L0/L1/L2 intent representations; its edge-cloud trigger chain reduces reporting volume to approximately 8.7%. Second, a Meta Engine uses a MetaModel for layered M1-to-M2-to-M3 reasoning: intent summarization, strategy planning informed by Strategy Memory, and parameter translation. It dispatches the resulting parameters through a unified outlet with safety guardrails. A Reward Dual Loop continuously optimizes both components by combining offline simulation for strategy-space exploration with online feedback for outcome calibration, forming a cycle of generation, execution, evaluation, and experience accumulation. Large-scale A/B tests on Taobao's homepage feed show that re-ranking control alone improves IPV by 2.06%, Core IPV by 2.39%, and GMV by 0.88%. Extending control to fine ranking raises these gains to 2.71%, 3.06%, and 1.31%, respectively, while consistently improving PV by more than 1%. These gains require neither replacement of pipeline models nor compromise of serving stability, supporting agentic meta-control as a viable paradigm for industrial recommendation.

DREAM Team

Figure 1 compares recommendation optimization paradigms.

  • Figure 1 presents a comparison of recommendation optimization paradigms.

1. Introduction

DREAM addresses fragmented, rigid, and weakly intent-aware recommendation pipelines by adding an autonomous policy layer with intent perception, strategy orchestration, and closed-loop optimization. Large-scale Taobao A/B tests report gains from controlling re-ranking and fine ranking without replacing pipeline models or compromising serving stability.

  • Industrial recommender pipelines fragment information and objectives, rely on rigid strategies, and inadequately track session-level intent shifts.
  • DREAM adds a perception-aware, orchestrable, and auditable policy layer over existing recommendation pipelines rather than replacing them.
  • The Intent Engine structures cross-domain on-device behavior into L0 Physical, L1 Demand, and L2 Preference representations while escalating only approximately 8.7% of behavior to cloud inference.
  • The Meta Engine performs M1 intent summarization, M2 strategy planning with Strategy Memory, and M3 parameter translation before guarded downstream dispatch.
  • Offline simulation and online feedback form a Reward Dual Loop that cycles through strategy generation, execution, evaluation, experience accumulation, and regeneration.
  • 2.06% IPV, 2.39% Core IPV, and 0.88% GMV gains arise from re-ranking control alone, increasing to 2.71%, 3.06%, and 1.31% with fine-ranking control.

2. DREAM Framework

DREAM overlays conventional recommendation stages with a closed-loop architecture that converts behavioral signals into structured intent, strategies, and bounded execution parameters. Its Intent Engine and Meta Engine are supported by unified dispatch, safety controls, and offline-online feedback.

  • 2. DREAM Framework: DREAM overlays conventional retrieval, ranking, and re-ranking with perception-aware strategy control and personalized execution parameters.
  • 2. DREAM Framework: The framework’s modules form a closed loop from behavioral signals to intent, decision, execution, and accumulated experience.
  • 2.1. Intent Engine: The Intent Engine supplies structured intent across temporal and semantic scales between behavioral data sources and recommendation applications.
  • 2.1. Intent Engine: The engine addresses missing real-time intent understanding through decoupled production and consumption, real-time updates, explainability, and progressive rollout.
  • 2.1. Intent Engine: The Intent Engine combines multi-source signals, change-point routing, incremental intent operations, and nightly Dreaming consolidation to manage online cost and long-term memory.
  • 2.2. Meta Engine: The Meta Engine treats downstream recommendation modules as controllable tools and uses the MetaModel to plan strategies from structured intent and real-time state.
  • 2.2. Meta Engine: Strategy Memory supplies positive references and negative constraints during planning, while the Unified Outlet routes instructions through one guarded dispatch path.
  • 2.2. Meta Engine: Offline simulation explores strategy space and online feedback calibrates strategies against live IPV, CVR, and GMV outcomes.

3. Intent Engine

The Intent Engine provides DREAM’s unified perception layer by compressing heterogeneous behavioral evidence into structured, hierarchical intent for downstream strategy control. Its cascaded reasoning and supply pipeline combines real-time inference, asynchronous refinement, and evaluation-backed evolution.

  • Intent reasoning: The Intent Reasoning Core combines a synchronous 0.8B Main Agent with routed asynchronous 4B specialists, self-evolution, and nightly Dreaming consolidation.The resulting perception-reasoning-supply pipeline feeds downstream applications and receives strategy-effectiveness feedback for refinement.
  • Multi-source data perception: The funnel addresses the trade-off between preserving cross-domain context and meeting latency, bandwidth, compute, and power constraints.On-device processing expands the behavior vocabulary from 6 to 60+ types, while change-point gating packages only about 15% of behavior for upload.
  • Multi-source data perception: The Traffic Funnel progressively filters, compresses, and enriches behavioral signals across four device-cloud stages before invoking intent reasoning.F1–F3 operate on device to encode, gate, and package behavior; F4 restores semantics and applies cloud-side admission control.
  • Intent representation: The Intent Engine converts explicit, implicit, and environmental signals into a structured L0 Physical, L1 Demand, and L2 Preference representation.L1 captures current needs, while L2 captures how users evaluate alternatives, including subcategory, brand, price, decision state, and real-time psychology.
  • Intent reasoning evaluation: 6.88 percentage points: routing increases the overall judge score, while self-evolution adds 6.54 percentage points and improves search-term recall by 4.37 percentage points.Filter recall decreases slightly by 0.39 percentage points but remains 2.15 percentage points above baseline.
  • Dreaming evaluation: Dreaming improves every evaluated intent field, with the largest gains in Intent Type, Priority, and Subcategory.Temporal node and Need Category improve only marginally because they are already reliable online.

4. Meta Engine

The Meta Engine converts structured intent into semantic strategies and guarded, stage-specific parameter overrides while preserving the existing recommendation pipeline. Intent-aware frequency control selectively invokes the LLM using context-dependent thresholds, and request-time orchestration applies cached strategies through deterministic processors.

  • Intent-aware frequency control: LLM invocation is treated as selective computation: trigger signals estimate when refreshing intent justifies invocation cost and latency.The policy uses behavioral change, normalized fused signals, and context-dependent thresholds; other events reuse cached or heuristic results.
  • Intent-aware frequency control: The detector fuses normalized drift, activity, and time signals into an urgency score for refreshing cached intent.Normalization uses group- and scenario-specific reference quantiles, while activity includes PV, UV, and high-value action counts.
  • Intent-aware frequency control: Thresholds combine global, user-group, time-bucket, scenario, and day-level corrections, with masks enforcing cooldowns, caps, allowlists, and overload protection.Illustrated thresholds are 0.45, 0.60, and 0.75 for high-, medium-, and low-value users.
  • Layered strategy reasoning: A shared strategy contract lets the MetaModel coordinate coupled decisions while stage-specific Tool Processors assign local parameter meanings.For example, one category preference can affect retrieval quota, ranking scatter, and re-ranking composition through different parameters.
  • Layered strategy reasoning: M1 summarizes user state, M2 produces schema-constrained semantic actions, and M3 translates them into operational parameters for downstream stages.Semantic actions include objective boosts, business support, preferences, experience constraints, and position policies.
  • Request-time orchestration: Request-time orchestration reads a cached strategy, validates it, and applies allowlisted local overrides while leaving defaults unchanged for missing, expired, or malformed bundles.The production pipeline remains unchanged, and processors dispatch relevant fields to retrieval, ranking, and re-ranking stages.

5. Experiments

DREAM is evaluated through cumulative production A/B tests, downstream Intent Engine tests, and a qualitative production trace. Results show gains from expanding Meta Engine control and from integrating intent outputs into recommendation applications.

  • Online Evaluation: The main online experiment uses a cumulative stage-wise ablation that extends Meta Engine control from re-ranking to fine ranking.The treatment configurations share the Intent Engine, MetaModel, parameter translation, and safety guardrails.
  • Online Evaluation: Re-ranking control improves PV by 1.03%, IPV by 2.06%, Core IPV by 2.39%, GMV by 0.88%, and PCTR by 0.76%.All reported metrics improve over the production baseline under re-ranking control alone.
  • Online Evaluation: Fine-ranking control raises IPV to +2.71%, Core IPV to +3.06%, and GMV to +1.31% over the production baseline.These correspond to additional gains of 0.65, 0.67, and 0.43 percentage points over re-ranking alone.
  • Online Evaluation: PV remains nearly unchanged between re-ranking and fine-ranking treatments, at +1.03% versus +1.04%.The reported discussion attributes deeper-stage gains primarily to improved engagement and conversion of existing exposure rather than additional exposure volume.
  • Intent Engine Downstream Evaluation: Intent-driven recall and strategy adaptation produce positive platform-wide lifts, while recall integration and copy optimization improve inquiry-card engagement within Heuristic Recommendation.Platform-wide Core IPV lifts are 0.91% and 0.70%; in-scenario inquiry-card clicks rise by 7.17% and 10.64%, respectively.
  • Case Study: The qualitative trace shows concurrent hierarchical intent cards feeding MetaModel summarization, strategy diagnosis, bounded control parameters, and product-sequence execution.The case includes multiple active intent cards, MetaModel-determined dominant and secondary intents, and sparse IPV/CTR-oriented overrides.

6. Conclusion

DREAM adds an intent-aware policy layer over conventional recommendation pipelines, combining structured perception, layered strategy translation, and dual-loop optimization. Taobao A/B tests report larger gains when control expands from re-ranking to fine ranking.

  • Conclusion: DREAM adds a perception-aware, orchestrable, and auditable policy layer atop retrieval–ranking–re-ranking pipelines without replacing existing models.Its Intent Engine and Meta Engine are continuously optimized through a Reward Dual Loop.
  • Conclusion: The Intent Engine fuses hundreds of on-device signals into L0/L1/L2 representations, while the Meta Engine translates intent into executable strategy bundles using layered reasoning and Strategy Memory.The Reward Dual Loop couples offline simulation with online feedback for continuous self-improvement.
  • Conclusion: Fine-ranking control raises gains to +2.71% IPV, +3.06% Core IPV, and +1.31% GMV, with PV consistently increasing by more than 1%.Re-ranking control alone achieves +2.06% IPV, +2.39% Core IPV, and +0.88% GMV.

A. Contributors

The contributor section lists the authors alphabetically by first name and identifies several contributors whose work was completed during a summer internship at Taobao & Tmall Group of Alibaba.

  • Contributors: The authors are listed alphabetically by first name.
  • Contributors: Several listed contributors are marked as having worked during a summer internship at Taobao & Tmall Group of Alibaba.The dagger annotation identifies the internship statement.

B.1. Policy Execution

The policy execution layer exposes controllable parameters while preserving a stable default configuration. M1 selects sparse strategy subsets and M3 translates them into bounded personalized overrides.

  • Policy Execution: L1 selects a sparse strategy subset per request, while L2 maps it to concrete values within guardrail-enforced safe ranges.
  • Policy Execution: The execution layer delivers Meta Engine decisions without replacing existing online pipeline modules.It combines default fallback with personalized overrides and safety guardrails.
  • Policy Execution: The exposed parameters include relevance-oriented knobs such as retrieval paths, ranking weights, and blending alpha, plus experience-oriented knobs such as diversity, fatigue, and filtering.M2 selects a sparse subset aligned with the strategy orientation, and M3 translates it into concrete values.

B.2. Offline MetaModel Policy Evaluation

The offline evaluation compares a 4B Base policy with replay RL under a fixed production-path setup, then specifies the intent and strategy-orchestration interfaces used by the system.

  • Offline MetaModel Policy Evaluation: Replay RL is compared with Base on identical held-out requests and execution components, isolating the contribution of replay RL.The comparison holds the strategy prompt, typed action schema, validator, compiler, execution engine, and production Evaluator fixed.
  • Offline MetaModel Policy Evaluation: pCTR rises 2.42%, pIPV rises 1.38%, and pGMV rises 0.37%, while pCVR changes by −0.99%.These are relative changes reported against the 4B Base policy.
  • Offline MetaModel Policy Evaluation: Validity improves 22.25% relatively, increasing from 80.86% to 98.85%, or 17.99 percentage points.Validity measures the fraction of generated strategy bundles that pass schema validation and compile into executable strategies.
  • Intent Engine: The Intent Engine infers L1 demand and L2 preference from available signals while prioritizing real-time over short-term, long-term, and static evidence.Unknown inputs are not assumed, and historical intent is refined or down-weighted according to new evidence.
  • Meta Engine: The Meta Engine outputs an M1 baseline and six M2 modules, including intent summary, ranking weights, card-type preferences, and category preferences.Ranking-weight levels range from −2 to +2 and adjust the scoring formula's objective weights.

D. Intent Engine Cases

The Intent Engine cases represent layered household-shopping intent using user profiles, behavioral history, and current product-comparison signals. The illustrated case centers on a newly emerged, high-confidence demand for a multifunctional food processor.

  • L0 Physical Layer: The case represents a 40–45-year-old female household-purchase decision-maker with relatively high purchasing power and multiple children.The profile also records coastal East China location, a high-end smartphone, and peak education spending.
  • L0 Physical Layer: The user has chronic interests spanning tea culture, fine tableware, outdoor sports gear, children’s education, collecting, household supplies, and wellness.Long-term behavioral memory also includes home renovation, ambient-lighting upgrades, and decorative details.
  • L1 Demand Layer: The active demand is a multifunctional food processor for family-kitchen meat grinding, blending, and other food-preparation tasks.The target audience is self-use for family cooking needs.
  • L1 Demand Layer: High-confidence comparison intent is supported by repeated “meat grinder” searches and seven product clicks across Brands A, B, and C within a concentrated period.The activity is described as a newly emerged demand in late February 2026.
  • L2 Preference Layer: The user prefers Brand A and favors multifunctionality, stainless-steel construction, large capacity, and high power, while remaining at competitive selection.No product has been added to the cart, and no product has been researched in depth.
Loading 2608.09408v3…