PaperlayerView exact revision on arXiv

Source-linked AI summary

Retail Product Search: A Practical Approach at Target

Darshan Sonagara, Qujiaheng Zhang, Ankit Singh, Alex Li

arXiv:2609.31498v1cs.IRcs.LG

TL;DR

Retail search must serve exact and open-ended shopping intents while balancing relevance, business goals, and low latency. Target combines lexical and vector retrieval with precision control and result fusion. Against lexical-only search, its best A/B-test variant improved engagement and business metrics and roughly halved zero-result searches.

  • Problem

    E-commerce search must handle varied shopping intents and heterogeneous product catalogs while accounting for business constraints such as inventory, profitability, and promotions.

    Source: §2.5 · p. 2

  • Method

    The system combines lexical and vector retrieval with engagement-derived training labels and a precision control module that filters vector candidates.

    Source: §1 · p. 1, §6 · p. 5

  • Results

    Against the lexical-only baseline, the best A/B-test variant raised CTR by 0.97%, order conversion by 0.98%, and demand per visitor by 1.10%, while cutting zero-result searches by about half.

    Source: §10 · p. 9

  • Takeaways & Limitations

    The hybrid search system is deployed on Target’s web and mobile platforms and serves millions of search queries each day.

    Source: §10 · p. 9

    The precision control module uses fixed NER-based and classifier-based filters with thresholds tuned offline, rather than adapting them to each query at inference time.

    Source: §11.3 · p. 9

Abstract

from arXiv · show

Search is one of the most important features in e-commerce, directly driving customer engagement and business growth. A good product search system must show both relevant and desirable results. However, retail search presents unique challenges. User intent can range from exact matches to open-ended discovery. Search systems must also balance multiple goals, such as relevance, revenue, and profit, while keeping response times low. Traditional keyword-based methods often fall short in handling natural language or semantic queries. Vector search helps alleviate these issues, but it can miss key intent signals or return low-precision results. In this paper, we present the design of a hybrid search system at Target that combines lexical and vector search. We describe our approach to data processing, embedding training, precision control for the final result set, multi-channel result fusion (where we compared fusion strategies and adopted weighted interleaving), and the performance optimizations used to maintain low latency for production deployment. Our method improves offline evaluation metrics, and in online A/B testing it raised click-through rate by 0.97%, order conversion by 0.98%, and demand per visitor by 1.10% over lexical-only search, while roughly halving zero-result searches. The resulting system is deployed at scale and serves millions of guests daily.

1 Introduction

Retail search must handle intent from exact requests to open-ended discovery while balancing relevance, business goals, and latency. This paper presents a production hybrid retrieval system and evaluates its components offline and online.

1 Introduction

  • Retail queries range from precise product requests to open-ended discovery, while search must balance relevance, sponsored items, revenue, and low latency.The paper notes that even small latency increases can hurt system quality and user outcomes at large scale.Source: §1 · p. 1
  • The paper focuses on retrieval within a multi-stage product-search pipeline, combining lexical and vector retrieval to be effective and fast.Later ranking stages reorder a broad candidate set using heavier models.Source: §1 · p. 1
  • Lexical retrieval can miss queries whose wording differs from catalog text, while dense retrieval can return loosely related items, miss exact matches, or add latency.The paper runs both channels on every query and merges their results.Source: §1 · p. 1
  • The system combines parallel lexical and fine-tuned dual-encoder retrieval, precision control of vector candidates, and weighted interleaving for result lists with little overlap.The paper also reports offline ablations and four-week online A/B tests measuring ranking quality, engagement, and conversion.Source: §1 · pp. 1–2
  • The paper's contributions also include training labels from engagement logs and precision filters based on attributes stated or predicted from queries.It compares reciprocal rank fusion with weighted interleaving and reports that weighted interleaving performed better in online tests.Source: §1 · pp. 1–2

2 Literature Review

The review traces lexical and semantic retrieval, industry embedding systems, and hybrid fusion approaches. It frames e-commerce search as requiring retrieval across varied shopping intents and business constraints.

2.1 Lexical Retrieval

  • Lexical methods such as TF-IDF and BM25 improve ranking but still require shared query-document terms; query expansion and relevance feedback can trade precision for recall and add noise.That trade-off matters in e-commerce, where users have little tolerance for irrelevant results.Source: §2.1 · p. 2

2.2 Semantic Search

  • Neural dense retrieval and sentence embeddings address vocabulary mismatch, but general-text embeddings may miss product relationships and purchase intent.The paper targets this gap with embeddings trained on engagement-derived labels.Source: §2.2 · p. 2

2.3 Embedding-Based Retrieval in Industry

  • Prior industry systems combine embedding retrieval with existing search infrastructure; the paper's closest comparison is Walmart's hybrid system for tail queries.The paper distinguishes its system by engagement-derived labels, vector precision control, and result-list merging.Source: §2.3 · p. 2

2.4 Hybrid Search Systems

  • Hybrid systems fuse parallel retrieval lists, filter one channel with another, or combine signals in learning-to-rank; each approach has trade-offs in calibration, overlap, and serving cost.RRF avoids score calibration but favors shared items, while LTR adds a re-ranking stage and latency.Source: §2.4 · p. 2

2.5 Challenges in E-Commerce Search

  • E-commerce search must serve intents from specific products to open-ended discovery across heterogeneous catalogs, while accounting for inventory, profitability, and promotions.The paper contributes a low-latency production system with engagement-derived training, vector precision control, and merging suited to lists with little overlap.Source: §2.5 · p. 2

3 Architecture

The architecture separates catalog indexing from query serving and runs lexical and vector retrieval in parallel. Precision control filters vector results before both result sets are merged for ranking.

3 Architecture

  • Catalog indexing embeds products and builds the vector index; at query time, parallel lexical and vector retrieval feed precision control and then ranking.The indexing and serving pipelines are separate.Source: §3 · p. 2, Figure 1 · p. 3
  • Solr lexical retrieval uses an inverted index, while an embedding service sends query vectors to an AlloyDB approximate-nearest-neighbor index using ScaNN.The lexical configuration supports spell correction, keyword rewriting, facet filtering, and custom boosts; the vector channel can match synonyms, abbreviations, and misspellings.Source: §3 · p. 3
  • A precision control module filters off-topic vector matches, and weighted interleaving merges the Solr and AlloyDB results before ranking.Channel weights are tuned through A/B tests.Source: §3 · p. 3
  • A separate indexing pipeline writes product embeddings to the AlloyDB index as catalog products are added or updated.The pipeline supports streaming and batch updates.Source: §3 · p. 3

4 Data

The training corpus is built from anonymized web and mobile interaction logs, with sampling intended to preserve category and query-frequency coverage. Engagement-derived labels include positives and two kinds of hard negatives.

4.1 Labeling

  • Training labels come from anonymized web and mobile interaction logs aggregated at the query-product level, without personally identifiable information.The logs were aggregated across platforms.Source: §4 · p. 3

4 Data

  • The training corpus contains about 20 million labeled query-product pairs, sampled to cover top-level categories, most subcategories, and head, torso, and tail query-frequency bands.The authors report that this sampling contributed substantially to performance on torso and tail queries.Source: §4 · p. 3

4.1.1 Positive Label Construction.

  • Positive labels use a weighted score of conversion, add-to-cart, and click signals, with stronger purchase-intent signals weighted more heavily.Pairs must remain above a threshold across multiple weeks; the weights and threshold are withheld, and clicks alone require a higher click rate than pairs with conversions.Source: §4.1 · p. 3, §4.1.1 · p. 3

4.1.2 Negative Label Construction.

  • Hard negatives include products shown often without engagement and products frequently retrieved near the top but not labeled positive.The two sources target ignored exposures and close lexical or semantic matches that do not satisfy the query; in-batch negatives are also used.Source: §4.1.2 · pp. 3–4

5 Semantic Model

The semantic model uses a fine-tuned dual encoder trained on product and query representations, with input masking, an improved contrastive loss, and staged training. e5-small-v2 was selected as the base encoder because its retrieval quality was close to larger models at much lower inference cost.

5.1.1 Pretrained Model Evaluation.

  • On the human-labeled benchmark, e5-large-v2 scored highest, but its 0.0068 NDCG gain over e5-small-v2 came at much higher inference cost, so e5-small-v2 was chosen for fine-tuning for strong retrieval quality and lower latency.Table 1 averaged NDCG over 5,000 queries with 200 candidates each.Source: §5.1.1 · p. 4, Table 1 · p. 4

5.1.2 Model Architecture.

  • A Siamese dual encoder shares weights across query and product inputs, mapping both into a common vector space for cosine-similarity retrieval.Product embeddings are precomputed offline and stored in an ANN index, while only the query is encoded at search time.Source: §5.1.2 · p. 4

5.2 Input Representation

  • Product attributes are concatenated with learned marker tokens, and random field masking reduces reliance on any single attribute when metadata is incomplete or noisy.Queries are encoded as short free-form text without field markers.Source: §5.2 · p. 4

5.3 Loss Function

  • Improved contrastive loss uses query-query and product-product similarities, as well as mismatched query-product pairs, as negatives within each batch.The full batch supplies negatives on both query and product sides, unlike a triplet loss that uses only one or a few negatives.Source: §5.3 · pp. 4–5

5.4 Training Strategy

  • Fine-tuning proceeds from random negatives to hard negatives, including repeatedly ignored products and near misses sharing surface features with the query.The hard-negative stage is intended to teach the model to reject apparently relevant products and improve precision near the top of the ranking.Source: §5.4 · p. 5

6 Precision Control

The precision control module filters vector candidates early to reduce off-target results, using inferred query intent and high-confidence constraints. It targets five product dimensions and separately limits exposure to sensitive or ambiguous categories.

Precision control module

  • The module combines a query classifier and Named Entity Recognition to extract explicit and implicit intent, then applies high-confidence filters across item type, category, brand, gender, and size group.The NER model captures explicit attributes such as brand or size, while the classifier infers item type and category.Source: §6 · p. 5

Sensitive-item filtering

  • Tiered item-type and category filters restrict sensitive or ambiguous products to searches that explicitly request them, reducing mismatches and helping maintain trust.For example, food searches should not return pet products unless the query asks for them.Source: §6 · p. 5

Serving and tuning

  • A standalone REST service generates filter parameters in real time for application to candidates before final ranking, with thresholds tuned to trade off precision against recall.The effect of the filters on retrieval quality is evaluated in Section 9.Source: §6 · p. 5

7 Merging Strategy

The paper compares Reciprocal Rank Fusion (RRF) with weighted interleaving for merging lexical and vector results. Because the channels overlap little and weighted interleaving outperformed RRF in online A/B tests, it was adopted in production.

7.1 Reciprocal Rank Fusion (RRF)

  • RRF combines channel rankings with smoothed rank contributions, rewarding items appearing in both lists; it works best when the lists overlap substantially.The smoothing constant was set to 60 in the experiments.Source: §7.1 · p. 5

7.2 Weighted Interleaving

  • Weighted interleaving builds a list one position at a time by randomly choosing a channel according to its global weight and adding that channel’s highest-ranked remaining item.It preserves each channel’s internal ordering, skips duplicates, and suits lists with little overlap.Source: §7.2 · p. 5
  • In online A/B tests, weighted interleaving outperformed RRF for the low-overlap lexical and vector channels, so it was selected for production.Channel weights were tuned through successive A/B tests and committed when key metrics showed a statistically significant positive lift (p < 0.05).Source: §7.2 · p. 5

8 Implementation Considerations

The implementation serves query embeddings through a dedicated service and supports production-scale traffic and catalog growth. CPU serving, caching, adaptive batching, and horizontal scaling help meet latency and load requirements.

8.1 Embedding Service Design

  • e5-small-v2 meets the production p99 latency requirement below 50 ms, with p75 and p95 latencies of roughly 25 to 30 ms.Its 384-dimensional embeddings also reduce vector-index memory use and serialization cost; further dimensionality reduction was not explored.Source: §8.1 · p. 6
  • The embedding service separates a lightweight application layer from a TorchServe model server, and the Figure 2 architecture includes an intervening cache and adaptive batching.The application layer handles routing, preprocessing, post-processing, and business logic; separating it from the model runtime makes model swaps or upgrades require few query-processing changes.Source: §8.1 · p. 6, Figure 2 · p. 6

8.2 Scalability

  • Kubernetes replication and autoscaling add serving pods as query volume rises, while the distributed ScaNN index handles millions of products and near-real-time updates.The ANN index is sharded, and asynchronous updates keep new or changed product embeddings current as the catalog changes.Source: §8.2 · p. 6

8.3 Infrastructure

  • Fine-tuning uses distributed NVIDIA A100 GPUs, while online and batch serving run on CPU instances to balance training performance with serving cost.The paper attributes CPU serving efficiency to the small e5-small-v2 model and TorchScript export.Source: §8.3 · p. 6

9 Experiments and Results

Offline ablations examine training data, modeling choices, and precision filters; live A/B tests compare hybrid search with lexical search and assess zero-result searches.

9.1 Offline Evaluation

  • Offline evaluation ranks roughly 200 candidates per query in an engagement-labeled set of more than 5,000 queries, rather than measuring full-catalog recall.The authors assume strong golden-set ranking also indicates full-catalog retrieval quality and note that online gains are consistent with this assumption.Source: §9.1 · pp. 6–7, §9 · p. 6

9.2 Data Processing Experiments

  • NDCG@5 rose from 0.8200 to 0.8661 and MRR from 0.9473 to 0.9760 when training expanded from 7M to 20M examples.Hard negatives then gave the best result on every metric, including NDCG@5 of 0.8794 and MRR of 0.9822.Source: §9.2 · p. 7, Table 2 · p. 7

9.3 Modeling Experiments

  • NDCG@5 rose from 0.7621 to 0.7957 and MRR from 0.9201 to 0.9469 when the improved contrastive loss replaced InfoNCE after one epoch.The best configuration, deployed as Finetuned v2, adds highlights and dynamic field masking to the field-token setup and performs best on every metric.Source: §9.3 · p. 7, Table 3 · p. 7

9.4 Precision Control Experiments

  • P@1 rose from 0.890 with no filters to 0.929 with NER and behavior filters, and P@10 rose from 0.836 to 0.885.Production uses the full filter set.Source: §9.4 · p. 7, Table 4 · p. 8

9.5 Online Evaluation

  • Weighted interleaving improved all five tracked metrics, while RRF produced mixed results; the team adopted weighted interleaving in production.Finetuned v2 later raised CTR by 0.97% and order conversion by 0.98% over the lexical baseline.Source: §9.5 · p. 8
  • CTR rose 0.97%, order conversion 0.98%, and demand per visitor 1.10% for Finetuned v2 over the lexical-only baseline.The deployed configuration combines expanded training data with hard negatives, field marker tokens, highlights with dynamic masking, and weighted interleaving.Source: §9.5 · p. 8, Table 5 · p. 8, §9.6 · p. 9

9.6 Zero-Result Rate Reduction

  • About 50% fewer zero-result searches occurred with hybrid search than with the lexical-only control over the four-week test.The vector channel can return related products without a lexical match, although precision filters can still remove every candidate.Source: §9.6 · p. 8

10 Conclusion

The conclusion presents a deployed hybrid retrieval system whose lexical and dense channels complement one another. It reports stronger live outcomes for the fine-tuned variant and highlights domain adaptation over model size.

10 Conclusion

  • The best A/B-test variant raised CTR by 0.97%, order conversion by 0.98%, and demand per visitor by 1.10%, while cutting zero-result searches by about half.The system is deployed on Target’s web and mobile platforms and serves millions of queries each day.Source: §10 · p. 9
  • Lexical retrieval remains strong for shared query-product terms, while dense retrieval finds relevant products when wording differs; weighted interleaving outperformed RRF in this setting.The paper attributes the fusion difference to the channels returning largely different items, weakening RRF’s overlap signal.Source: §10 · p. 9
  • Domain adaptation mattered more than model size: moving from e5-small-v2 to e5-large-v2 added less than 0.01 NDCG on the benchmark, while fine-tuned models gained online.Caching, adaptive batching, and ANN indexing kept embedding service latency within budget on CPUs; precision control constrained vector results to query attributes.Source: §10 · p. 9

11 Future Work

Future work targets production-observed limitations through personalized retrieval, multimodal and additional retrieval channels, and learned precision control.

11.1 Personalized Retrieval

  • Personalized retrieval could condition query embeddings on browsing and purchase history, but would need diversity-aware reranking to limit discovery loss for new or infrequent shoppers.Source: §11.1 · p. 9

11.2 Multimodal and Multi-Channel Retrieval

  • Text-and-image embeddings could enable cross-modal retrieval, while behavioral co-occurrence and structured attribute matching could add recall-broadening channels to the merging framework.Source: §11.2 · p. 9

11.3 Learned Precision Control

  • A lightweight learned relevance scorer could replace fixed NER/classifier thresholds, handle ambiguous or multi-intent queries better, and reduce false positives and false negatives.Source: §11.3 · p. 9

Related papers

Loading 2609.31498v1…