PaperlayerView exact revision on arXiv

Source-linked AI summary

EAGER: Enrich-and-Align Generative Query Recommendation from Clicked Items in E-commerce Search

Shuwei Yuan, Mingqian Ding, Luxin Liu, Rong Xiao, Xiaoyi Zeng

arXiv:2609.07143v1cs.IR

TL;DR

Historical-log methods have limited reach for long-tail and personalized intents, while vanilla LLMs can produce generic queries disconnected from user preferences. EAGER combines intent-enriching SFT with GRPO alignment, and offline evaluations and online A/B tests report gains in engagement and payment conversion.

  • Problem

    Log-based methods depend on historical behavior and predefined query pools, while vanilla LLMs can produce generic queries that lack personalization and priority ordering.

    Source: §1 · pp. 1–2

  • Method

    EAGER uses SFT with multi-source intent supervision to expand query coverage, then GRPO with hybrid rule-based and preference-aware click rewards to align generation with deployment constraints.

    Source: §1 · p. 2

  • Results

    Offline evaluations and online A/B tests show substantial improvements in user engagement and payment conversion.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Production deployment and A/B tests report gains in engagement and downstream conversion, with intent coverage and deployment alignment treated as complementary objectives.

    Source: §5 · p. 7

    The method is developed and evaluated in one real-world e-commerce search scenario, and generalization to other search or recommendation scenarios requires further validation.

    Source: Limitations · p. 7

Abstract

from arXiv · show

E-commerce platforms increasingly display clickable query suggestions alongside items in the user feed, enabling users to refine or expand their intent without manually reformulating queries. Existing approaches either mine suggestions from historical logs -- limited to past behavior and blind to long-tail, personalized intents -- or rely on off-the-shelf LLMs whose lack of platform-specific knowledge yields fluent but generic queries disconnected from real click behavior. We propose EAGER (Enrich-and-AliGn gEnerative Query Recommendation), a two-stage framework for generating query suggestions from clicked items. In the enrichment stage, supervised fine-tuning (SFT) follows a four-stage curriculum that scales information richness (from item-only to user-conditioned) and reasoning depth (from direct to chain-of-thought). Each stage incorporates rationale augmentation, diversity regularization, and self-distillation. In the alignment stage, we post-train via GRPO with a hybrid reward of multiple rule-based business signals and a preference-aware click reward. Extensive offline experiments and online A/B test demonstrate the effectiveness of EAGER, which has been deployed in production at a major e-commerce platform.

1 Introduction

E-commerce query recommendation helps users express shopping intent, and item-to-query recommendation offers suggestions after a user clicks an item. EAGER addresses the limits of log-based methods and generic LLM outputs with a two-stage framework for generating and aligning queries.

1 Introduction

  • Log-based methods rely on historical co-occurrence and predefined query pools, limiting fine-grained personalization and long-tail or new-item coverage.The authors report a positive correlation between query-pool richness and downstream conversion.Source: §1 · p. 1
  • Clicked-item query recommendation must cover multiple latent intents from one explicit choice while balancing click-through, conversion, format, and diversity objectives.Vanilla LLMs can produce fluent but generic queries without personalization or priority ordering.Source: §1 · p. 2
  • EAGER first uses SFT with multi-source intent supervision, curriculum learning, and self-distillation to broaden query coverage, then uses GRPO with hybrid rewards to align queries with click preferences and deployment constraints.The two stages are described as complementary: SFT expands intent coverage, while GRPO aligns generation with online criteria.Source: §1 · p. 2
  • The framework generates personalized query sets from clicked items and user context, and is deployed at a large-scale e-commerce platform.A shared Preference-Aware Reward Model anchors both training stages on the same production click signal.Source: §1 · p. 2
  • Offline evaluations and online A/B tests show EAGER substantially improves user engagement and payment conversion.Source: §1 · p. 2

2 Preliminary

The paper studies user-contextualized query generation in the item-to-query setting, where a clicked item, user context, and serving constraints inform a set of generated queries.

2 Preliminary

  • The model takes a recently clicked item, user context, and online serving constraints as inputs and generates a query set of size K.Source: §2 · p. 2
  • Unlike pool-based retrieval, item-to-query generation produces queries directly from item semantics and immediate user behavior rather than selecting from a predefined inventory.Source: §2 · p. 2
  • Generated queries are subject to accuracy, diversity, and online deployability constraints.Source: §2 · p. 2

3 Methodology

EAGER uses supervised fine-tuning to enrich query generation, followed by hybrid-reward GRPO alignment with online recommendation objectives. The SFT pipeline builds intent-rich supervision, trains through increasingly rich tasks, and expands query diversity; alignment combines business-rule and user-click signals.

3.1 Supervised Fine-Tuning for Query Enrichment

  • SFT enriches query generation from clicked items and user context, covering latent intents and creating a richer candidate space for later reinforcement learning and ranking.The framework separates offline query enrichment from reinforcement learning for online alignment.Source: §3.1 · p. 2, §3 · p. 2

3.1.1 Multi-source Intent-ordered Supervision

  • Intent-ordered supervision combines clicked queries, users’ shortly post-click queries, and LLM-generated queries scored by PARM to prioritize their quality.Clicked queries provide the strongest supervision; PARM also supplies the model-based reward in GRPO.Source: §3.1.1 · pp. 2–3

3.1.2 Rationale-Augmented Curriculum Learning

  • The four-task curriculum progresses from item-only query generation to user-conditioned generation, adding teacher-generated rationales to guide the model’s reasoning.Enhanced I2Q and Enhanced UI2Q incorporate rationales; UI2Q adds personalized user information.Source: §3.1.2 · pp. 3–4

3.1.3 Diversity-aware SFT Enhancement

  • Diversity-aware SFT adds a regularization term that penalizes excessive similarity among queries, then self-distills filtered, high-quality sampled outputs into additional training data.Sampling raises temperature and top-p; generated queries are filtered for invalid formats, abnormal lengths, and business-rule violations.Source: §3.1.3 · p. 4

3.2 Reinforcement Learning with Hybrid Rewards

  • GRPO post-training with hybrid rewards aligns generation with online I2Q objectives beyond offline SFT supervision.The hybrid reward combines rule-based business constraints with a model-based click-preference reward.Source: §3.2 · p. 4, §3.2.3 · p. 5

3.2.1 Verifiable Rule-Based Reward

  • The rule-based reward is a weighted sum of eight verifiable signals addressing user experience, compliance, and fluency, enforcing display requirements and business constraints.These rewards reduce format errors and redundant generation.Source: §3.2.1 · p. 4

3.2.2 Model-based Reward via PARM

  • PARM estimates query click likelihood in item-user context and supplies GRPO’s model-based reward; its score also ranks LLM-generated queries during SFT data construction.The per-query reward is one minus the probability of predicting “no,” and set-level reward averages query rewards.Source: §3.2.2 · p. 4

3.2.3 GRPO Policy Optimization

  • GRPO optimizes a weighted combination of rule-based and model-based rewards to guide query lists toward business constraints and real user click preferences.The same preference-aware signal connects SFT supervision selection and GRPO reward optimization.Source: §3.1.1 · p. 3, §3.2.3 · p. 5

4 Experiment

Experiments assess EAGER’s relevance, diversity, and compliance offline, then measure engagement and conversion online. Ablations examine its training components and hyperparameter choices.

4.1 Experimental Setup

  • Offline evaluation measures relevance, diversity, and compliance, while online evaluation reports engagement and downstream conversion metrics.Soft HR@K counts exact logged-query matches or queries with at least 60% of their terms in the clicked item's title after search; baselines include open- and closed-source LLMs.Source: §4.1 · p. 5

4.2 Offline Experiments

  • EAGER-base achieves the highest Hard HR and Distinct-2, while full EAGER trades a moderate Hard HR decrease for gains in Soft HR@1, APD, and Rule score.Closed-source LLMs achieve competitive Soft HR but relatively weak Hard HR; zero-shot open-source models perform poorly across metrics. EAGER-4B has the highest APD and EAGER-8B the best Soft HR.Source: §4.2 · pp. 5–6

4.3 Ablation Study

  • Ablations show diversity regularization trades some Soft HR@1 for higher APD and Distinct-2, while GRPO chiefly improves Rule compliance and curriculum subtasks add relevance and diversity.Without DIV, Soft HR@1 is 78.34 vs. 77.96, while APD is 68.24 vs. 72.96 and Distinct-2 is 78.86 vs. 83.04. Adding rationale gives +3.17 APD, user context gives +4.06 Soft HR@1, and the full curriculum contributes +6.57 Soft HR@1 and +6.97 APD.Source: §4.3 · p. 6, Table 3 · p. 6, Table 5 · p. 6

4.4 Hyper-parameter Analysis

  • Performance peaks around λ = 0.1; self-distillation improves as temperature rises, peaking around 0.9–1.0, with larger top-k consistently performing better.Increasing λ further degrades relevance, indicating overly strong diversity regularization hurts the balance between diversity and accuracy.Source: §4.4 · p. 7, Figure 3 · p. 7

4.5 Online Experiments

  • Online, UCTR and PCTR rise by 0.83% and 1.28%, while DAC, Pay Count, and L2P increase by 2.98%, 3.49%, and 2.38%.All reported gains have p < 0.05 and remain stable throughout the 9-day test.Source: Table 4 · p. 6, §4.5 · p. 7

5 Conclusion

EAGER generates personalized query recommendations from clicked items, bridging item recommendation and search. Its two stages address intent coverage and deployment alignment in sequence, with production deployment and A/B tests reporting gains in engagement and downstream conversion.

  • EAGER generates personalized query recommendations from clicked items, bridging item recommendation and search.Source: §5 · p. 7
  • SFT expands the generation space for diverse user intents, while GRPO constrains it to meet online business requirements without sacrificing diversity.Source: §5 · p. 7
  • Production deployment and A/B tests confirm consistent gains in user engagement and downstream conversion.Source: §5 · p. 7

Limitations

The paper notes platform-specific generalization and feedback-bias limits, manually designed rewards, and the absence of joint downstream optimization. Related work spans query recommendation and generative recommendation, while EAGER targets clicked-item I2Q in natural language.

Limitations

  • The method is developed and evaluated in real-world e-commerce search, so generalization to other search or recommendation scenarios requires further validation.PARM also relies on historical exposure and click logs that may inherit position and exposure bias; manually designed rule-based rewards may need adaptation as business requirements change, and the framework does not jointly optimize downstream ranking or conversion.Source: Limitations · p. 7

A Related Work

  • Prior query-recommendation methods condition on user-typed queries, whereas I2Q generates a personalized query set from a clicked item alone.The reviewed methods include statistical, neural, and LLM-based approaches.Source: A Related Work · p. 9
  • EAGER generates natural-language queries without discrete indexing, unlike semantic-ID approaches to generative recommendation.Source: A Related Work · p. 9

B PARM Training Details

PARM predicts whether a candidate query will be clicked in an item-user context and serves both offline query ranking and GRPO reward. Its training uses position-aware click-log samples and generative yes/no supervision.

B PARM Training Details

  • PARM ranks offline LLM-generated queries during SFT data construction and supplies the model-based reward for GRPO optimization.Source: B PARM Training Details · p. 9

Data Construction

  • Position-aware sampling discards first-position clicks, treats the clicked query as positive and preceding queries as negative, and ignores queries after the click.The strategy is intended to reduce positional bias and avoid treating insufficiently examined queries as negatives.Source: B PARM Training Details · p. 9

Training Strategy

  • PARM uses clicked item information, user personalization features, and a candidate query to generate a natural-language yes/no click label, trained with autoregressive negative log-likelihood.The formulation lets the model reuse LLM semantic understanding to learn matching relationships among the inputs.Source: B PARM Training Details · pp. 9–10

Offline Ranking Agreement

  • Offline ranking agreement compares PARM’s top-k query sets with the production ranker’s top-k sets for each exposure event.Source: B PARM Training Details · p. 10
  • PARM achieves a 79.8% top-1 match rate with the production ranker, and top-2 to top-4 match rates remain above 72%, indicating consistent alignment beyond the first position.Source: B PARM Training Details · p. 10

C Rule Reward

The rule-based rewards enforce I2Q experience and business requirements using deterministic offline checks. They cover query quality, quantity, item relevance, user intent, fluency, diversity, and prohibited content.

C Rule Reward

  • Rule-based rewards use deterministic rules and linguistic features offline, without online services or additional model-based evaluation.Source: C Rule Reward · p. 10
  • Experience compliance penalizes harmful or unnatural expressions, while quantity compliance checks whether the output contains the required number of queries.Source: C Rule Reward · p. 10
  • Lexical overlap checks query terms against the item title, while user-intent alignment rewards keywords from the user’s historical behavior.For queries shorter than three words, at least two words must match the title; otherwise, at least 75% of query terms should be covered.Source: C Rule Reward · pp. 10–11
  • Fluency uses linguistic features, and diversity rewards lower Jaccard similarity among generated queries.Source: C Rule Reward · p. 11
  • Additional penalties target blacklisted, sensitive, brand-infringing, unsafe, or low-quality content; Figure 5 contrasts EAGER’s fine-grained preferences with the baseline’s generic queries.Source: C Rule Reward · p. 11, Figure 5 · p. 11

D Evaluation Metrics

The section defines offline measures of accuracy, relevance, diversity, and rule compliance, alongside online engagement and conversion metrics. Together, these measures cover query generation quality and the user funnel from exposure to purchase.

D Evaluation Metrics

  • Offline evaluation covers accuracy, relevance, diversity, and rule compliance, while online evaluation measures engagement and downstream conversion.The online metrics span query exposure through payment.Source: D Evaluation Metrics · pp. 11–12
  • Hard HR@K counts exact string matches to the clicked query, with HardHR@1 evaluating the first generated query and HardHR@All the full set.A hit occurs when any generated query exactly matches the ground truth.Source: D Evaluation Metrics · p. 11
  • Soft HR counts exact matches or queries with at least 60% of their terms in the clicked result item's title, capturing semantic intent beyond exact matching.The section presents this as better reflecting whether a query covers the intent behind the final clicked item.Source: D Evaluation Metrics · p. 11
  • Distinct-2 measures the ratio of unique 2-grams to all 2-grams, while APD averages pairwise query dissimilarity; higher values indicate greater diversity.APD defines pairwise distance as 1−sim(qi, qj).Source: D Evaluation Metrics · pp. 11–12
  • Rule is a weighted sum of business-rule scores covering experience, quantity, lexical overlap, user-intent alignment, and penalties; higher scores mean better compliance.Source: D Evaluation Metrics · p. 12
  • Online metrics separate engagement (UCTR and PCTR) from conversion (DAC, Pay Count, and L2P) across the funnel from exposure to payment.UCTR measures users clicking at least one suggestion; PCTR measures clicked suggestions per exposure, while DAC, Pay Count, and L2P track purchase outcomes.Source: D Evaluation Metrics · p. 12

E Implementation Details

The implementation fine-tunes Qwen3-1.7B, then uses the resulting model as the initial policy for GRPO. In production, EAGER generates candidates as a third recall channel, which a shared ranker scores alongside existing retrieval sources.

E Implementation Details

  • SFT fine-tunes Qwen3-1.7B for one epoch with full-parameter updates, a 4096-token maximum sequence length, batch size 64, learning rate 2e-5, and bf16 precision on 24 NVIDIA A100 80GB GPUs.Training uses the LLa-MAFactory framework.Source: E Implementation Details · p. 12
  • The SFT model initializes GRPO in ROLL, where each rollout response is scored using verifiable rules and PARM.Source: E Implementation Details · p. 12
  • EAGER is a third recall channel beside log-based and dense retrieval; a shared context-feature ranker scores merged candidates and returns the top-4 queries.EAGER expands long-tail coverage where co-click statistics are sparse and supplies candidates that compete under the same ranker on head items.Source: Figure 6 · p. 12, E Implementation Details · p. 12
  • Online inference uses vLLM to generate 10 independent complete results, with compute and VRAM proportional to their total token volume.The passage says this creates less pressure than Beam Search because the candidates are independent.Source: E Implementation Details · p. 12

F Case Study

The case study shows EAGER generating more specific luggage queries using a user's click history and demographic attributes. The prompt templates provide product and user context and specify query-generation criteria and output requirements.

F Case Study

  • For a South Korean user interested in large-capacity and designer-style luggage, EAGER generated specific queries, unlike the generic tote-bag examples attributed to traditional methods.Examples include “large capacity tote bag” and “designer PU leather handbag.”Source: F Case Study · pp. 12–13

G Prompt Templates

  • The prompt instructs the model to infer intent and scenarios, avoid fabricated attributes, keep queries relevant and diverse, and use at most six words.It also specifies the output language and prohibits unsafe or prohibited content.Source: G Prompt Templates · p. 13
  • The output specification requires one valid JSON object containing a language field and generated query texts.Source: G Prompt Templates · p. 13
  • The SFT prompt supplies product details and user profile and behavior, directing generation of 10 distinct, native, high-intent queries.Inputs include demographics, search history, click history, and click-category history.Source: G Prompt Templates · p. 13

Related papers

Loading 2609.07143v1…