PaperlayerView exact revision on arXiv

Source-linked AI summary

SAM-D2Q: Aligning Multimodal Doc2Query with Search Demand and Conversion for E-commerce

Hui Zhou, Jian Hui Ji, Lei Ma, Rong Xiao, Xiaoyi Zeng

arXiv:2609.04961v1cs.IR

TL;DR

E-commerce search suffers from vocabulary mismatch because short product titles omit diverse user expressions and visual attributes, while traditional Doc2Query is text-only and not business-aligned. SAM-D2Q addresses this with multimodal fine-tuning, counterfactual augmentation, and reinforcement-learning preference alignment under Boolean retrieval constraints. In AliExpress Search, it improved online business metrics, including GMV and Pay Count.

  • Problem

    Traditional Doc2Query methods may produce commercially ineffective expansions and miss visual product attributes because they rely mainly on text and semantic relevance.

  • Method

    SAM-D2Q combines multimodal supervised fine-tuning, counterfactual visual augmentation, and GRPO-based preference alignment for Boolean e-commerce retrieval.

  • Results

    3.38% GMV lift and 2.27% Pay Count increase were achieved in AliExpress Search online A/B testing.

  • Takeaways & Limitations

    SAM-D2Q generates expansion queries that are semantically relevant, visually grounded, and commercially valuable for large-scale industrial search.

Abstract

from arXiv · show

E-commerce search often suffers from vocabulary mismatch between user queries and merchant-authored product titles, since short titles cannot fully cover diverse user expressions or visual product attributes. Although Doc2Query alleviates this issue by generating pseudo-queries for document expansion, traditional methods are text-only and not optimized for e-commerce business objectives. As a result, they may produce semantically plausible but commercially ineffective expansions and miss key attributes present in product images. To this end, we propose E-commerce Search-Aligned Multimodal Doc2Query (SAM-D2Q), a business-aligned multimodal document expansion framework for e-commerce search under Boolean retrieval constraints. SAM-D2Q consists of three stages: (1) task-adapted multimodal supervised fine-tuning to enhance vision-language understanding of product titles, images, and user queries; (2) multimodal data augmentation to improve perception of key visual attributes and expansion coverage; and (3) reinforcement-learning-based preference alignment toward search business objectives, encouraging the model to generate pseudo-queries that better match user intent and commercial value. Offline experiments show that SAM-D2Q substantially improves retrieval performance over traditional Doc2Query methods. Deployed in the AliExpress production search system, SAM-D2Q improves online business metrics, increasing GMV by +3.38% and Pay Count by +2.27%.

1 Introduction

E-commerce search still faces vocabulary mismatch because product titles are short and text-only expansion misses visual attributes and business value. SAM-D2Q addresses these gaps with multimodal, business-aligned Doc2Query and reports improved retrieval and online metrics.

  • Keyword-based retrieval remains central to large-scale e-commerce search, but user queries and product descriptions often use different terms for the same intent.
  • Traditional Doc2Query may generate semantically related terms with limited commercial value because it overlooks search demand and conversion signals.Examples include commercially weaker variants such as “chic dress” compared with attribute-specific queries such as “sexy dress” or “casual dress.”
  • Text-only expansion can miss image-expressed attributes such as color, sleeve length, pattern, and material, especially for products with sparse titles.
  • SAM-D2Q combines multimodal information, data synthesis, and business-aligned optimization to generate product-grounded queries valuable for commercial search performance.
  • SAM-D2Q uses information-gain-constrained multimodal SFT, counterfactual visual augmentation, and GRPO-based preference alignment as three successive stages.
  • 3.38% GMV lift and 2.27% Pay Count increase were achieved in AliExpress Search online A/B testing.

2 Related Work

Prior Doc2Query research expands documents with generated queries, but e-commerce requires multimodal, commercially aligned expansion because product titles are incomplete and visual attributes matter. Recent methods address noise filtering, multimodal bias, and objective alignment through relevance filtering, counterfactual augmentation, and online reinforcement learning.

  • Document expansion: Doc2Query methods generate synthetic queries and append them to documents to alleviate vocabulary mismatch before retrieval.Early Doc2Query uses a Seq2Seq Transformer, while DocT5Query uses a larger T5 model.
  • Document expansion: Doc2Query-- filters generated queries with a relevance model to reduce irrelevant or hallucinated terms.
  • E-commerce search: E-commerce expansion must account for short product titles, visually expressed attributes, and commercial value beyond semantic correctness.These requirements arise within recall-focused industrial search systems.
  • Multimodal understanding: Multimodal models can over-rely on language priors and ignore visual evidence, motivating counterfactual and augmented training samples.Prior approaches perturb inputs, mask critical objects or words, or pair questions using semantic annotations.
  • Preference alignment: The framework adapts GRPO with semantic, commercial, and visual rewards to align query generation with retrieval effectiveness and business utility.This addresses limitations of costly or unstable RLHF and static offline preference methods.

3 Preliminaries

E-commerce retrieval relies on Boolean matching because deterministic intent handling, concise product data, and large-scale latency requirements make general probabilistic retrieval unsuitable. Consequently, document expansion must add precise new terms rather than repeat existing tokens, motivating the Boolean-aware Information-Gain Constraint.

  • Boolean retrieval: Industrial e-commerce search relies heavily on Boolean Matching rather than probabilistic models such as BM25.The passage identifies deterministic matching, data conciseness, and latency constraints as the operational reasons.
  • Boolean retrieval: Boolean retrieval supports strict intent-slot precision, structured product-title matching, and computational efficiency at very large scale.
  • Boolean-aware expansion: Under Boolean constraints, expansion must produce high-precision, business-aligned terms that comply with Boolean retrieval logic.
  • Boolean-aware expansion: Repeating an existing title token provides zero marginal recall utility because Boolean matching treats term presence as binary.Standard Doc2Query also uses repetition for re-weighting, but that function is redundant here.
  • Boolean-aware expansion: The generative expander is therefore valued for producing semantically relevant latent keywords absent from seller descriptions.The Information-Gain Constraint uses non-covered query-item pairs for lexical expansion and text-covered pairs for counterfactual visual augmentation.

4 Methodology

SAM-D2Q formulates multimodal e-commerce Doc2Query as conditional query generation under Boolean retrieval, using a three-stage curriculum that progresses from information-gain-constrained lexical expansion to visual grounding and business-utility alignment.

  • Task formulation: SAM-D2Q learns a conditional generation policy over product text and images to infer user intents that are semantically implied but lexically absent.
  • Stage 1: Lexical expansion: Stage 1 retains query-item logs containing terms not fully covered by product text, discouraging token extraction and encouraging informative expansion queries.The policy is trained on DpreSFT, while text-covered pairs are reserved for later visual augmentation.
  • Stage 2: Visual grounding: Stage 2 combines CPV-guided counterfactual augmentation with filtered human-labeled data to improve visual grounding while preserving information gain.Visual attribute values are masked from product text while product images remain unchanged, requiring recovery of missing visual information.
  • Stage 2: Visual grounding: The masked-input training task requires the model to recover original queries from product images when visual attributes have been removed from text.This removes textual shortcuts while retaining the product image as evidence for the missing attributes.
  • Stage 2: Visual grounding: The hybrid Stage-2 corpus combines multimodal augmentation and filtered human supervision, avoiding simple text copying while learning cross-modal grounding and semantic relevance.
  • Stage 3: Business alignment: Stage 3 uses GRPO to optimize a gated composite reward for semantic correctness, commercial value, and visual grounding.The gate assigns zero reward when relevance or useful-term introduction checks fail, preventing popular but irrelevant queries from exploiting business signals.

5.1 Experimental Setup

The evaluation uses AliExpress production data, a large inverted-index testbed, frozen multimodal relevance labels, and neural document-expansion baselines to assess generation and retrieval quality.

  • Training Data: 9 million valid query-item pairs support Stage 1 coarse semantic alignment, while Stage 2 adds 500k counterfactual samples for visual grounding.The alignment stage also uses 3.5 million human-annotated pairs and 80k RL exploration prompts.
  • Evaluation Environment: The offline test uses an 8.8 million-item inverted index and 30,000 live-traffic queries stratified into head, torso, and tail segments.Evaluation omits the online active-product deployment filter.
  • Evaluation Environment: A frozen production Query-Item evaluator uses product text and images, with relevance determined by a threshold calibrated on held-out human labels.The evaluator is fixed before testing and is not trained on generated test queries.
  • Baselines: Offline comparisons isolate multimodal SFT, counterfactual augmentation, and RL alignment using 8B-scale neural document-expansion variants.The baselines include original titles, text-only Doc2Query, and multimodal SFT; traditional expansion methods are evaluated online.
  • Metrics: Generation quality uses Relevance and Query-Value, retrieval effectiveness uses # Rel. Items and Total Qual., and online impact uses CTR, CVR, Pay Count, and GMV.Total Qual. combines relevant retrieved items with fixed item-level business quality scores based on historical efficiency signals.

5.2 Offline Evaluation

Offline results show that multimodal grounding substantially improves generation relevance, while alignment increases commercial value and SAM-D2Q improves retrieval quality across cutoffs and search modules.

  • Generation Quality: 43.11% relevance for text-only expansion rises by +27.3 p.p. when multimodal signals are introduced.The text-only baseline exhibits semantic drift attributed to lacking visual grounding.
  • Generation Quality: Stage 3 adds only +0.4 p.p. relevance but raises Commercial Value to 0.981, indicating distinct roles for SFT and RL alignment.The reported interpretation is that SFT improves semantic plausibility while RL alignment optimizes commercial utility.
  • Retrieval Performance: Adding SAM-D2Q to query rewriting improves Top-3000 relevance and quality, while concatenating its terms also improves dense retrieval.These results support complementarity between query-side rewriting, document-side expansion, and embedding-based retrieval.

5.3 Online Deployment and Evaluation

SAM-D2Q was deployed through offline indexing and evaluated in AliExpress Search with online A/B testing. The deployment improved core business metrics and reduced zero-result search traffic.

  • Deployment setup: SAM-D2Q was evaluated in a 21-day A/B test on 4% of live traffic against the existing production stack, adding only document-side expansion.The treatment kept other retrieval and ranking components unchanged.
  • Deployment setup: The enhanced index generated filtered candidate queries offline, avoiding online model inference while supporting daily incremental updates.Generated queries were inserted into the inverted index after relevance and safety filtering.
  • Online results: +3.38% GMV and +2.27% Pay Count were achieved on overall search traffic.UV Value, CTR, and CVR also increased by +2.98%, +0.25%, and +0.53%, respectively.
  • Online results: The zero-result PV ratio fell 17%, from 2.07% to 1.72%, indicating improved sparse-retrieval coverage.A reverse-bucket validation also reported declines after SAM-D2Q was disabled.

6 Conclusion

SAM-D2Q is a multimodal document-to-query framework for industrial e-commerce search. It combines multimodal training, counterfactual data synthesis, and business-aware alignment, with offline and online results supporting its effectiveness and deployment feasibility.

  • Contribution: SAM-D2Q bridges vocabulary mismatch in sparse retrieval by generating semantically relevant, visually grounded, commercially valuable expansion queries.The framework targets industrial e-commerce search.
  • Method: The framework combines multimodal SFT, counterfactual data synthesis, and business-aware GRPO alignment with semantic, commercial, and visual rewards.These components support multimodal and business-aligned query expansion.
  • Results: Offline experiments improved generation quality and retrieval effectiveness, while online AliExpress Search testing improved core business metrics including GMV and Pay Count.The conclusion also reports deployment feasibility in large-scale industrial search systems.
Loading 2609.04961v1…