PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM4AIGQ: LLM-based AI Guidance Query Generation Framework for Multi Interest Mining

Xiangchen Pan, Jiayi Xu, Jing Wang, Xing Fang, Lingyun Zhu

arXiv:2609.03674v1cs.IR

TL;DR

Traditional Q2AIGQ pipelines can suffer semantic drift and insufficient multi-interest intent modeling. LLM4AIGQ disentangles interests from user profiles and histories, generates an AIGQ for each sub-interest, and combines SFT, RL, DPO, and nearline serving; it performs strongly offline and online and is deployed at hundreds-of-millions-user scale.

  • Problem

    Traditional Q2AIGQ generation suffers semantic drift from information-cascade loss and relies heavily on user–item co-occurrence, limiting multi-interest intent modeling.

  • Method

    LLM4AIGQ integrates user profiles and historical interactions to disentangle sub-interests, generate corresponding AIGQs, and train through SFT, RL, and DPO with nearline generation and online retrieval.

  • Results

    LLM4AIGQ outperforms zero-shot state-of-the-art and larger same-family models in relevance and shopping guidance value, with a 4.46% uCTR gain over baseline and 2.53% gain at 40% traffic.

  • Takeaways & Limitations

    The framework supports multi-interest-aware guidance-query generation and has been deployed online for hundreds of millions of users.

Abstract

from arXiv · show

Guidance queries stimulate user consumption by extracting preferences to provide search queries with guidance value, playing a crucial role in the e-commerce field. Traditional AI-generated queries (AIGQ) generation primarily relies on a two-stage "Query-to-AI-Generated-Query" (Q2AIGQ) association paradigm, first recalling user primary search queries from user profiles, historical behavior sequences, item-side information, and the current query through multi-path retrieval, then generalizing AIGQ via rule-based methods. This approach suffers from semantic drift due to information cascade loss; additionally, primary search query derivation heavily depends on "user-item" co-occurrence relationships, lacking exploration of user multi-interests, resulting in guidance queries with low value and mismatched purchase intent. To address the expressive limitations of traditional co-occurrence-based retrieval, we propose LLM4AIGQ, an LLM-based solution for generating AI guidance queries tailored to users' multi-interests. This approach segments user interests by integrating user profiles and historical interaction sequences, infers specific consumption intents for each sub-interest, and subsequently generates corresponding AIGQ. In terms of model training, we employ a post-training pipeline comprising Supervised Fine-Tuning (SFT), Reinforcement Learning (RL), and Direct Preference Optimization (DPO) to enhance the model's capability in generating AIGQ. We also introduce a multi-level reward design to satisfy the requirements of multi-objective optimization and long-chain reasoning in practical applications. Regarding deployment, we adopt a nearline-generation and online-read architecture to meet latency constraints. Extensive experimental analyses demonstrate that our model achieves robust performance in both offline evaluations and online A/B tests.

1 Introduction

LLM4AIGQ addresses semantic drift and weak multi-interest modeling in traditional guidance-query pipelines by generating AIGQs directly from disentangled user interests. Its SFT–RL–DPO training and nearline deployment target query quality, reasoning, and latency.

  • Challenges: Traditional Q2AIGQ pipelines retrieve primary queries through multiple paths before rule-based AIGQ generation, creating information-cascade loss and semantic drift.The retrieval inputs include user profiles, interaction histories, and item-side information.
  • Motivation: LLM-based generation infers shopping guidance intent from profiles and historical behaviors to improve personalization and alignment with purchase intentions.The approach is motivated by LLMs’ semantic understanding and text-generation capabilities.
  • Challenges: Users’ noisy historical interactions contain diverse interests, requiring the model to disentangle sub-interests before generating corresponding guidance queries.Noise may arise from accidental interactions and exposure bias.
  • Training: SFT learns AIGQ style from single-interest data, while RL uses multi-objective rewards to supervise interest disentanglement and query generation.The training pipeline is designed to address both style learning and mixed-interest reasoning.
  • Training and Deployment: DPO optimizes reasoning expression, and nearline generation with online retrieval reduces latency during serving.The deployment design combines offline and nearline query generation with online retrieval.

2 Related Work

Related work progresses from query–item matching and profile-based personalization toward models that represent sequential dependencies and multiple user interests. LLMs increasingly support semantic representation, instruction tuning, and recommendation generation.

  • Query Recommendation: Traditional query recommendation emphasizes query–item relevance, while later methods incorporate user profiles and historical search behaviors.Examples include semantic similarity, user embeddings, attention aggregation, and sequential encoders.
  • Multi-Interest Modeling: Some recommendation studies explicitly model multiple interests using knowledge-graph embeddings or structural patterns in user–query–item graphs.CAMI and GraphSRRL are cited as examples of multi-interest modeling.
  • LLM-Based Recommendation: LLMs have been applied to recommendation through semantic representation enhancement, instruction tuning, retrieval augmentation, and reward-driven reinforcement learning.The cited examples include LLMRec, ReFICR, and Rec-R1.

3 Preliminary

LLM4AIGQ formulates guidance-query generation as identifying multiple user interests from profile and historical context, then producing an AIGQ for each sub-interest. Its framework combines compressed behavioral representation with staged training and deployment components.

  • Data Composition: The LLM input comprises task instructions and user context, with context formed from a user profile and historical behavior sequence.The profile summarizes consumption patterns and basic personal information, while the sequence contains recent behaviors.
  • Task Objective: The task requires identifying user interests and generating concise, compliant AIGQs that address purchase pain points and provide shopping-guidance value.Generated queries should align with users’ consumption patterns.
  • Framework Overview: The overview framework uses an LLM to generate item short titles, compressing behavioral content before SFT, RL, DPO, and nearline–online serving.The figure description assigns style learning to SFT, interest disentanglement to RL, reasoning-expression optimization to DPO, and retrieval to deployment.

4 Methodology

LLM4AIGQ uses a three-stage SFT–RL–DPO methodology: SFT learns AIGQ style, RL handles interest segmentation and intent inference, and DPO transfers think-mode reasoning to non-think inference.

  • SFT: SFT teaches AIGQ style through supervised learning on single-interest scenarios.Single-interest training focuses generation on query style rather than choosing among multiple interests.
  • RL: RL trains the LLM to segment interests and infer consumption intent under complex, mixed-interest scenarios.The stage addresses multi-interest reasoning beyond isolated subsequences.
  • DPO: DPO distills think-mode reasoning capability so the model can produce high-quality outcomes during non-think inference.Preference optimization is used to preserve reasoning effectiveness without explicit reasoning traces at inference.
  • Deployment: Deployment uses nearline generation and online retrieval to serve guidance queries with lower online latency.The architecture separates query generation from online serving.

4.1 Data Process and Context Compression

The data process compresses historical behavior and normalizes item information before training. It retains recent interactions and uses LLM-generated short titles to make behavioral context more concise.

  • Historical behavior compression: Only the most recent 50 interaction behaviors are retained to capture short-term interests and reduce computational overhead.Each behavior records an index, relative time, behavior type, and either item-title or search-query content.
  • Item-title compression: Item titles are summarized from item, CPV, and brand-category information to remove redundant marketing and flagship-store keywords.
  • Item-title compression: A larger-capacity model’s short titles serve as standard answers for fine-tuning the item short-title task.
  • Item-title compression: The short-title task maps item information to concise query text reflecting core selling points and user concerns.

4.2 SFT Training: AIGQ Style Learning in Single Interest

SFT teaches the model what constitutes a concise, compliant, guidance-valued AIGQ by training on single-interest subsequences. Interest segmentation separates mixed historical behavior before style learning.

  • AIGQ style learning: SFT instruction tuning teaches AIGQ definitions, application scenarios, guidance value, content compliance, and concise expression.Target scenarios include single-item consultation, comparative decision-making, and category selection.
  • Motivation: Directly fine-tuning on composite-interest histories can cause autonomous extrapolation, weak low-frequency-trigger handling, and candidates concentrated on one main interest.
  • Single-interest construction: A teacher LLM segments each historical sequence into sub-interests, extracts chronological subsequences, and infers corresponding guidance queries.
  • Single-interest construction: Single-interest training encourages the model to learn guidance-query generation style rather than choose among multiple interests.
  • Optimization: The SFT objective uses the user profile, model parameters, and generated AIGQ length as its core training variables.

4.3 RL Training: Interest Division and AIGQ Generation

RL trains the model to segment mixed interests, infer consumption intent, and generate multiple compliant queries under hierarchical rewards. Rewards assess reasoning, query quality, within-interest diversity, and cross-interest diversity.

  • Interest division and generation: The RL prompt requires interest summarization, consumption-intent inference with next-item prediction, and three confidence-scored guidance queries per subsequence.
  • Interest division and generation: At most three interest subsequences are formed using temporal proximity, semantic relevance, and profile preferences.
  • Interest division and generation: The model outputs inferred interest labels and corresponding guidance queries in JSON format after the reasoning process.
  • Hierarchical rewards: Think-level rewards supervise interest segmentation and mining accuracy by comparing generated and reference subsequences and labels.
  • Hierarchical rewards: Reasoning-format reward equals 1 only when ["Step1", "Step2", "Step3"] appear and all required JSON attributes are complete.
  • Hierarchical rewards: Query-level rewards enforce a maximum of 15 Chinese characters and score commercial guidance value through a manually trained judge model.
  • Hierarchical rewards: Interest-level rewards combine semantic accuracy, Type-Token Ratio stylistic diversity, and exact candidate-count formatting within each interest cluster.
  • Hierarchical rewards: Global content-diversity reward represents each interest group by mean-pooled query embeddings and penalizes semantic similarity across groups.

4.4 DPO Training: Distill the Reasoning Ability of Think Mode

DPO distills reasoning learned in think mode into direct final-answer generation because removing explicit reasoning otherwise causes conditional distribution shift and lower guidance-query quality. Preference-pair filtering further improves DPO effectiveness.

  • Motivation: Removing deep reasoning noticeably deteriorates guidance-query quality because training and inference output formats differ.The discrepancy creates conditional distribution shift and degraded inference performance.
  • DPO Design: DPO uses think-mode final answers as positive samples and non-thinking outputs as negative samples for preference alignment.The method retains final answers while discarding explicit reasoning traces; 70% of negatives come from RL checkpoints and 30% from the base model.
  • DPO Design: Filtering out the top 30% most semantically similar preference pairs improves the effectiveness of DPO training.The filtering strategy targets insufficient preference margins and ineffective training.

4.5 Online Deployment

LLM4AIGQ separates nearline generation from online retrieval to balance guidance-query accuracy with high-throughput, low-latency serving. Nearline updates personalized candidates after accumulated interactions, while online matching and ranking serve them in real time.

  • Serving Architecture: The serving design uses nearline generation and online lightweight matching and ranking to balance accuracy and responsiveness.The nearline pipeline generates and stores candidate guidance queries, while the online pipeline processes real-time requests.
  • Nearline Pipeline: After every N accumulated interactions, the nearline pipeline generates multiple interest labels and guidance queries and writes them to the mapping table.Examples of accumulated interactions include clicks or favorites, and generation operates in non-thinking mode.
  • Online Pipeline: The online pipeline uses an offline user-to-guidance-query mapping table and efficient vector retrieval to recall personalized candidates in real time.The mapping table dynamically receives results from the nearline pipeline when users enter the shopping-guide interface.

5 Experiment

Offline experiments and online evaluations assess LLM4AIGQ’s relevance, shopping-guidance value, efficiency, reward design, preference filtering, and staged training effects. Results show improvements over zero-shot and model variants, with measurable online gains and trade-offs between quality and inference efficiency.

  • 5.2 Model Performance: LLM4AIGQ outperforms zero-shot models and larger same-family models on relevance and shopping guidance value.The comparison uses Taobao test-set relation and value metrics.
  • 5.2 Model Performance: Each training stage improves performance: SFT learns AIGQ style, RL improves interest mining and query quality, and DPO transfers reasoning to non-thinking inference.The reported gains span style, interest disentanglement, relevance, value, and non-thinking reasoning.
  • 5.2 Model Performance: Single-interest tuning consistently outperforms mixed-interest tuning because mixed-interest data contain greater noise and information dilution.The advantage persists through later training stages.
  • 5.3.2 The Effectiveness of Multi-Level Rewards in GRPO: Removing any reward component decreases both relevance and shopping-guidance value, while different rewards affect interest disentanglement, query quality, relevance, diversity, and cross-interest separation.Think-level rewards primarily affect relevance, query-level rewards affect value, and interest- and global-level rewards support semantic correctness and diversity.
  • 5.3.3 The Effectiveness of Data Filtering in DPO: Removing DPO preference-pair filtering significantly reduces relevance and shopping-guidance value because semantically similar pairs destabilize optimization and encourage homogeneous outputs.The filtering strategy removes the top 30% most semantically similar positive-negative pairs.
  • 5.5 Online A/B Test: A 4.46% uCTR improvement at 10% traffic remained a 2.53% gain after scaling to 40% traffic over six days.The online tests covered shopping guidance, search, and item detail scenarios in Tmall.

6 Conclusion

The proposed LLM-based framework addresses semantic drift and weak personalization in shopping guidance query generation, autonomously disentangling user interests and generating queries aligned with consumption intentions.

  • LLM4AIGQ targets semantic drift and limited personalized intent capture in traditional shopping guidance query pipelines.
  • The framework autonomously disentangles user interests and generates shopping guidance queries tailored to users’ consumption intentions.
  • The approach has been deployed online at the scale of hundreds of millions of users.
Loading 2609.03674v1…